high_scalability high_scalability-2008 high_scalability-2008-257 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Update: Final Thoughts on SSD and MySQL AKA Battleship Spinn3r . Tips on how to make your database 10x faster using solid state drives. Potential exists for 100x speedup. Solid-state drives (SSDs) are the holy grail of storage. The promise of RAM speeds and hard disk like persistence have for years driven us crazy with power user lust, but they've stayed tantalizingly just out of reach. Always too expensive, too small, and oddly too slow. Has that changed? Can you now miraculously have your cake and eat it too? Can you now have it both ways? Is balancing work with family life now as easy as tripping over a terabyte drive? In a pioneering series of blog articles Kevin Burton conducts original research on next generation SSD drives in real world configurations. For an experience report on his great adventure you can turn to: Could SSD Mean a Rise in MyISAM Usage? , Serverbeach, MySQL and Mtron SSDs , Prediction: SSD Blades in 2008 , Zeus IOPS - Another High
sentIndex sentText sentNum sentScore
1 Solid-state drives (SSDs) are the holy grail of storage. [sent-4, score-0.297]
2 The promise of RAM speeds and hard disk like persistence have for years driven us crazy with power user lust, but they've stayed tantalizingly just out of reach. [sent-5, score-0.302]
3 Always too expensive, too small, and oddly too slow. [sent-6, score-0.115]
4 Can you now miraculously have your cake and eat it too? [sent-8, score-0.19]
5 Is balancing work with family life now as easy as tripping over a terabyte drive? [sent-10, score-0.299]
6 In a pioneering series of blog articles Kevin Burton conducts original research on next generation SSD drives in real world configurations. [sent-11, score-0.221]
7 For an experience report on his great adventure you can turn to: Could SSD Mean a Rise in MyISAM Usage? [sent-12, score-0.211]
8 , More SSD vs HDD vs InnoDB vs MyISAM Numbers . [sent-15, score-0.441]
9 Unfortunately Goldilocks may still find the porridge too slow. [sent-17, score-0.125]
10 SSDs turn out to be fast, but not as fast as you might hope in out of the box configurations. [sent-18, score-0.1]
11 MySQL on SSD is fast for sequential reads and writes, but random reads are writes are relatively slow. [sent-19, score-0.576]
12 Kevin speculates "Log structured filesystems can come into play here and seriously increase performance by turning all random writes into sequential writes" and that "Bigtable and append only databases would FLY on flash. [sent-20, score-0.894]
13 " The upshot for me is we need a storage engine designed specifically for SSDs as they provide a very different design space from hard disks. [sent-21, score-0.106]
14 Kevin has a lot of excellent details and observations on his site and he'll no doubt be coming up with a lot more. [sent-22, score-0.162]
wordName wordTfidf (topN-words)
[('ssd', 0.447), ('kevin', 0.234), ('myisam', 0.23), ('ssds', 0.205), ('writes', 0.161), ('vs', 0.147), ('crazy', 0.141), ('sequential', 0.133), ('speculates', 0.133), ('lust', 0.133), ('maria', 0.133), ('tripping', 0.133), ('porridge', 0.125), ('goldilocks', 0.125), ('suspicious', 0.119), ('drives', 0.118), ('mysql', 0.116), ('oddly', 0.115), ('adventure', 0.111), ('cake', 0.108), ('blades', 0.108), ('filesystems', 0.108), ('random', 0.106), ('upshot', 0.106), ('pioneering', 0.103), ('turn', 0.1), ('findings', 0.099), ('grail', 0.099), ('burton', 0.097), ('aka', 0.096), ('append', 0.094), ('hdd', 0.094), ('stayed', 0.089), ('terabyte', 0.089), ('reads', 0.088), ('observations', 0.087), ('innodb', 0.083), ('seriously', 0.083), ('eat', 0.082), ('holy', 0.08), ('family', 0.077), ('final', 0.077), ('turning', 0.076), ('doubt', 0.075), ('rise', 0.075), ('tips', 0.074), ('exists', 0.074), ('fly', 0.073), ('speeds', 0.072), ('thoughts', 0.072)]
simIndex simValue blogId blogTitle
same-blog 1 1.0 257 high scalability-2008-02-22-Kevin's Great Adventures in SSDland
Introduction: Update: Final Thoughts on SSD and MySQL AKA Battleship Spinn3r . Tips on how to make your database 10x faster using solid state drives. Potential exists for 100x speedup. Solid-state drives (SSDs) are the holy grail of storage. The promise of RAM speeds and hard disk like persistence have for years driven us crazy with power user lust, but they've stayed tantalizingly just out of reach. Always too expensive, too small, and oddly too slow. Has that changed? Can you now miraculously have your cake and eat it too? Can you now have it both ways? Is balancing work with family life now as easy as tripping over a terabyte drive? In a pioneering series of blog articles Kevin Burton conducts original research on next generation SSD drives in real world configurations. For an experience report on his great adventure you can turn to: Could SSD Mean a Rise in MyISAM Usage? , Serverbeach, MySQL and Mtron SSDs , Prediction: SSD Blades in 2008 , Zeus IOPS - Another High
2 0.32949847 1291 high scalability-2012-07-25-Vertical Scaling Ascendant - How are SSDs Changing Architectures?
Introduction: With Amazon announcing new High I/O 2TB SSD instances the age of SSD has almost arrived. I say almost because the $27K a year price tag for the hi1.4xlarge on demand instance type is outside the budget of many. Yet even at the full on demand rate the price per IOP for the high IO instance is attractive: 27 cents ($27K/100K IOPS) per vs $1.25 for disk . With the obvious benefits of giant SSD machines combined with 10 Gbps networking, it’s interesting to consider: what architecture decisions might you make differently in the future? More Headroom for Vertical Scaling Simplifies Everything The beauty of higher hardware performance is it shifts effort away from the programmer which allows developers to focus on the business of business, minimizing trickeration. This has always been the allure of vertical scaling and is well realized by SSDs through a combination of high throughput, low latencies, and just as important, high densities. We have a few earl
3 0.26613003 1066 high scalability-2011-06-22-It's the Fraking IOPS - 1 SSD is 44,000 IOPS, Hard Drive is 180
Introduction: Planning your next buildout and thinking SSDs are still far in the future? Still too expensive, too low density. Hard disks are cheap, familiar, and store lots of stuff. In this short and entertaining video Wikia's Artur Bergman wants to change your mind about SSDs. SSDs are for today, get with the math already. Here's Artur's logic: Wikia is all SSD in production. The new Wikia file servers have a theoretical read rate of ~10GB/sec sequential, 6GB/sec random and 1.2 million IOPs. If you can't do math or love the past, you love spinning rust. If you are awesome you love SSDs. SSDs are cheaper than drives using the most relevant metric: $/GB/IOPS. 1 SSD is 44,000 IOPS and one hard drive is 180 IOPS. Need 1 SSD instead of 50 hard drives. With 8 million files there's a 9 minute fsck. Full backup in 12 minutes (X-25M based). 4 GB/sec random read average latency 1 msec. 2.2 GB/sec random write average latency 1 msec. 50TBs of SSDs in one machine for $80,000. With the densi
4 0.15117611 980 high scalability-2011-01-28-Stuff The Internet Says On Scalability For January 28, 2011
Introduction: Submitted for your reading pleasure... Something we get to say more often than you might expect - funny NoSQL comic: How to Write a CV (SFW) Playtomic shows hows how to handle over 300 million events per day, in real time, on a budget . More Speed, at $80,000 a Millisecond . Does latency matter ? Oh yes... “On the Chicago to New York route in the US, three milliseconds can mean the difference between US$2,000 a month and US$250,000 a month.” Quotable Quotes @jkalucki : Throwing 1,920 CPUs and 4TB of RAM at an annoyance, as you do. @jointheflock @hkanji : Scale can come quick and come hard. Be prepared. @elenacarstoiu : When you say #Cloud, everybody's thinking lower cost. Agility, scalability and fast access are advantages far more important. @BillGates : From Melinda - Research proves we can save newborn lives at scale Kosmix with a fascinating look at Cassandra on SSD , summarizing some of what they've learned over the past year runni
5 0.12660621 1369 high scalability-2012-12-10-Switch your databases to Flash storage. Now. Or you're doing it wrong.
Introduction: This is a guest post by Brian Bulkowski , CTO and co-founder of Aerospike , a leading clustered NoSQL database, has worked in the area of high performance commodity systems since 1989. Why flash rules for databases The economics of flash memory are staggering. If you’re not using SSD, you are doing it wrong. Not quite true, but close. Some small applications fit entirely in memory – less than 100GB – great for in-memory solutions. There’s a place for rotational drives (HDD) in massive streaming analytics and petabytes of data. But for the vast space between, flash has become the only sensible option. For example, the Samsung 840 costs $180 for 250GB. The speed rating for this drive is rated by the manufacturer at 96,000 random 4K read IOPS, and 61,000 random 4K write IOPS. The Samsung 840 is not alone at this price performance. A 300GB Intel 320 is $450. An OCZ Vertex 4 256GB is $235, with the Intel being rated as slowest, but our internal testing showing
6 0.1242312 1183 high scalability-2012-01-30-37signals Still Happily Scaling on Moore RAM and SSDs
7 0.11741424 1475 high scalability-2013-06-13-Busting 4 Modern Hardware Myths - Are Memory, HDDs, and SSDs Really Random Access?
9 0.11503924 1093 high scalability-2011-08-05-Stuff The Internet Says On Scalability For August 5, 2011
10 0.11205935 967 high scalability-2011-01-03-Stuff The Internet Says On Scalability For January 3, 2010
11 0.10741648 734 high scalability-2009-10-30-Hot Scalabilty Links for October 30 2009
12 0.10614932 554 high scalability-2009-04-04-Digg Architecture
13 0.105975 1511 high scalability-2013-09-04-Wide Fast SATA: the Recipe for Hot Performance
14 0.10495238 1281 high scalability-2012-07-11-FictionPress: Publishing 6 Million Works of Fiction on the Web
15 0.10172392 1186 high scalability-2012-02-02-The Data-Scope Project - 6PB storage, 500GBytes-sec sequential IO, 20M IOPS, 130TFlops
16 0.09579657 1169 high scalability-2012-01-05-Shutterfly Saw a Speedup of 500% With Flashcache
17 0.09485738 805 high scalability-2010-04-06-Strategy: Make it Really Fast vs Do the Work Up Front
18 0.093671963 1487 high scalability-2013-07-05-Stuff The Internet Says On Scalability For July 5, 2013
19 0.090939924 1261 high scalability-2012-06-08-Stuff The Internet Says On Scalability For June 8, 2012
20 0.089104623 327 high scalability-2008-05-27-How I Learned to Stop Worrying and Love Using a Lot of Disk Space to Scale
topicId topicWeight
[(0, 0.127), (1, 0.065), (2, -0.032), (3, 0.009), (4, -0.018), (5, 0.085), (6, -0.047), (7, 0.025), (8, 0.056), (9, -0.037), (10, -0.009), (11, -0.112), (12, 0.001), (13, 0.073), (14, -0.025), (15, 0.007), (16, -0.028), (17, 0.03), (18, -0.046), (19, 0.05), (20, 0.03), (21, -0.006), (22, -0.091), (23, 0.053), (24, -0.044), (25, 0.011), (26, 0.01), (27, -0.031), (28, -0.034), (29, -0.056), (30, 0.021), (31, -0.045), (32, 0.073), (33, 0.03), (34, -0.013), (35, 0.007), (36, -0.039), (37, -0.001), (38, 0.024), (39, -0.122), (40, -0.004), (41, -0.006), (42, 0.044), (43, 0.049), (44, 0.037), (45, -0.079), (46, 0.029), (47, -0.043), (48, 0.003), (49, -0.069)]
simIndex simValue blogId blogTitle
same-blog 1 0.97564119 257 high scalability-2008-02-22-Kevin's Great Adventures in SSDland
Introduction: Update: Final Thoughts on SSD and MySQL AKA Battleship Spinn3r . Tips on how to make your database 10x faster using solid state drives. Potential exists for 100x speedup. Solid-state drives (SSDs) are the holy grail of storage. The promise of RAM speeds and hard disk like persistence have for years driven us crazy with power user lust, but they've stayed tantalizingly just out of reach. Always too expensive, too small, and oddly too slow. Has that changed? Can you now miraculously have your cake and eat it too? Can you now have it both ways? Is balancing work with family life now as easy as tripping over a terabyte drive? In a pioneering series of blog articles Kevin Burton conducts original research on next generation SSD drives in real world configurations. For an experience report on his great adventure you can turn to: Could SSD Mean a Rise in MyISAM Usage? , Serverbeach, MySQL and Mtron SSDs , Prediction: SSD Blades in 2008 , Zeus IOPS - Another High
2 0.81360048 1066 high scalability-2011-06-22-It's the Fraking IOPS - 1 SSD is 44,000 IOPS, Hard Drive is 180
Introduction: Planning your next buildout and thinking SSDs are still far in the future? Still too expensive, too low density. Hard disks are cheap, familiar, and store lots of stuff. In this short and entertaining video Wikia's Artur Bergman wants to change your mind about SSDs. SSDs are for today, get with the math already. Here's Artur's logic: Wikia is all SSD in production. The new Wikia file servers have a theoretical read rate of ~10GB/sec sequential, 6GB/sec random and 1.2 million IOPs. If you can't do math or love the past, you love spinning rust. If you are awesome you love SSDs. SSDs are cheaper than drives using the most relevant metric: $/GB/IOPS. 1 SSD is 44,000 IOPS and one hard drive is 180 IOPS. Need 1 SSD instead of 50 hard drives. With 8 million files there's a 9 minute fsck. Full backup in 12 minutes (X-25M based). 4 GB/sec random read average latency 1 msec. 2.2 GB/sec random write average latency 1 msec. 50TBs of SSDs in one machine for $80,000. With the densi
3 0.70277178 1291 high scalability-2012-07-25-Vertical Scaling Ascendant - How are SSDs Changing Architectures?
Introduction: With Amazon announcing new High I/O 2TB SSD instances the age of SSD has almost arrived. I say almost because the $27K a year price tag for the hi1.4xlarge on demand instance type is outside the budget of many. Yet even at the full on demand rate the price per IOP for the high IO instance is attractive: 27 cents ($27K/100K IOPS) per vs $1.25 for disk . With the obvious benefits of giant SSD machines combined with 10 Gbps networking, it’s interesting to consider: what architecture decisions might you make differently in the future? More Headroom for Vertical Scaling Simplifies Everything The beauty of higher hardware performance is it shifts effort away from the programmer which allows developers to focus on the business of business, minimizing trickeration. This has always been the allure of vertical scaling and is well realized by SSDs through a combination of high throughput, low latencies, and just as important, high densities. We have a few earl
4 0.66512167 1511 high scalability-2013-09-04-Wide Fast SATA: the Recipe for Hot Performance
Introduction: This is a guest post by Brian Bulkowski , CTO and co-founder of Aerospike , a leading clustered NoSQL database, has worked in the area of high performance commodity systems since 1989. This blog post will tell you exactly how to build a multi-terabyte high throughput datacenter server. A fast, reliable multi-terrabyte data tier can be used for recent behavior (messages, tweets, plays, actions), or anywhere that today you use Redis or Memcache. You need to know: Which SSDs work Which chassis work How to configure your RAID cards Intel’s SATA solutions – combined with a high capacity storage server like the Dell R720xd and a host bus adapter based on the LSI 2208, and a Flash optimized database like Aerospike , enables high throughput and low latency. In a wide configuration, with 12 to 20 drives per 2U server, individual servers can cost-effectively serve at high throughput with 16T at $2.50 per GB with the s3700, or $1.25 with the s3500. Other SSD of
5 0.65815407 1384 high scalability-2013-01-09-The Story of How Turning Disk Into a Service Lead to a Deluge of Density
Introduction: We usually think of the wonderful advantages of service oriented architectures as a software thing, but it also applies to hardware. In Security Now 385 , that Doyen of Disk, Steve Gibson , tells the fascinating story (@ about 41:30) of how moving to a service oriented architecture in hard drives, modeling a drive as a linear stream of sectors, helped create the amazing high density disk drives we enjoy today. When drives switched to use the IDE (integrated drive electronics) interface, the controller function moved into the drive instead of the computer. No longer were low level drive signals moved across cables and into the motherboard. Now we just ask the drive for the desired sector and the drive takes care of it. This allowed manufacturers to do anything they wanted to behind the IDE interface. The drive stopped being dumb, it became smart, providing a sort of sector service. Density sky rocketed because there was no dependency on the computer. All the internals could co
6 0.64607918 980 high scalability-2011-01-28-Stuff The Internet Says On Scalability For January 28, 2011
7 0.62152261 1475 high scalability-2013-06-13-Busting 4 Modern Hardware Myths - Are Memory, HDDs, and SSDs Really Random Access?
8 0.60114318 1129 high scalability-2011-09-30-Stuff The Internet Says On Scalability For September 30, 2011
9 0.59713304 1114 high scalability-2011-09-13-Must see: 5 Steps to Scaling MongoDB (Or Any DB) in 8 Minutes
10 0.58792412 829 high scalability-2010-05-20-Strategy: Scale Writes to 734 Million Records Per Day Using Time Partitioning
11 0.58482963 1322 high scalability-2012-09-14-Stuff The Internet Says On Scalability For September 14, 2012
12 0.58378237 1247 high scalability-2012-05-18-Stuff The Internet Says On Scalability For May 18, 2012
13 0.58033592 104 high scalability-2007-10-01-SmugMug Found their Perfect Storage Array
14 0.57460403 729 high scalability-2009-10-28-And the winner is: MySQL or Memcached or Tokyo Tyrant?
15 0.56658405 1183 high scalability-2012-01-30-37signals Still Happily Scaling on Moore RAM and SSDs
16 0.56584579 1352 high scalability-2012-10-31-Gone Fishin': LiveJournal Architecture
17 0.56182092 1588 high scalability-2014-01-31-Stuff The Internet Says On Scalability For January 31st, 2014
18 0.56099492 3 high scalability-2007-07-09-LiveJournal Architecture
19 0.55490202 1407 high scalability-2013-02-15-Stuff The Internet Says On Scalability For February 15, 2013
20 0.55349088 1369 high scalability-2012-12-10-Switch your databases to Flash storage. Now. Or you're doing it wrong.
topicId topicWeight
[(1, 0.099), (2, 0.15), (10, 0.193), (61, 0.049), (77, 0.017), (79, 0.1), (85, 0.067), (90, 0.202), (94, 0.01), (99, 0.023)]
simIndex simValue blogId blogTitle
same-blog 1 0.88110387 257 high scalability-2008-02-22-Kevin's Great Adventures in SSDland
Introduction: Update: Final Thoughts on SSD and MySQL AKA Battleship Spinn3r . Tips on how to make your database 10x faster using solid state drives. Potential exists for 100x speedup. Solid-state drives (SSDs) are the holy grail of storage. The promise of RAM speeds and hard disk like persistence have for years driven us crazy with power user lust, but they've stayed tantalizingly just out of reach. Always too expensive, too small, and oddly too slow. Has that changed? Can you now miraculously have your cake and eat it too? Can you now have it both ways? Is balancing work with family life now as easy as tripping over a terabyte drive? In a pioneering series of blog articles Kevin Burton conducts original research on next generation SSD drives in real world configurations. For an experience report on his great adventure you can turn to: Could SSD Mean a Rise in MyISAM Usage? , Serverbeach, MySQL and Mtron SSDs , Prediction: SSD Blades in 2008 , Zeus IOPS - Another High
2 0.8092106 1380 high scalability-2013-01-02-Why Pinterest Uses the Cloud Instead of Going Solo - To Be Or Not To Be
Introduction: I love it when a somewhat older article like Pinterest Cut Costs From $54 To $20 Per Hour By Automatically Shutting Down Systems hits Hacker News and generates a good conversation . One of the common sentiments about the cloud was raised: why doesn't Pinterest save a lot of money by running their own hardware instead of using the cloud? Ryan Park , Operations Engineer at Pinterest, responds in what I think is the perfect modern response to that ultimate existential design dilemma: Our #1 requirement has been to keep up with the growth in traffic on the site. We've been growing so fast that there's literally no way we could have ordered and racked equipment fast enough. We were also a very small team -- a year ago there were only about a dozen people in the whole company. At this point we're much larger, which gives us room to consider more options like colo or multiple cloud providers. AWS certainly feels pretty costly when you compare colo prices to the list price for o
3 0.80289721 1046 high scalability-2011-05-23-Evernote Architecture - 9 Million Users and 150 Million Requests a Day
Introduction: The folks at Evernote were kind enough to write up an overview of their architecture in a post titled Architectural Digest . Dave Engberg describes their approach to networking, sharding, user storage, search, and some other custom services. Evernote is a cool application, partially realizing Vannevar Bush 's amazing vision of a memex . Wikipedia describes Evernote's features succinctly: Evernote is a suite of software and services designed for notetaking and archiving. A "note" can be a piece of formattable text, a full webpage or webpage excerpt, a photograph, a voice memo, or a handwritten "ink" note. Notes can also have file attachments. Notes can then be sorted into folders, tagged, annotated, edited, given comments, and searched. Evernote supports a number of operating system platforms (including Android, Mac OS X, iOS, Microsoft Windows and WebOS), and also offers online synchronization and backup services. Key here is that Evernote stores a lot of data, that m
4 0.79933685 1635 high scalability-2014-04-21-This is why Microsoft won. And why they lost.
Introduction: My favorite kind of histories are those told from an insider's perspective. The story of Richard the Lionheart is full of great battles and dynastic intrigue. The story of one of his soldiers, not so much. Yet the soldiers' story, as someone who has experienced the real consequences of decisions made and actions taken, is more revealing. We get such a history in Chat Wars , a wonderful article written by David Auerbach, who in 1998 worked at Microsoft on MSN Messenger Service, Microsoft’s instant messaging app (for a related story see The Rise and Fall of AIM, the Breakthrough AOL Never Wanted ). It's as if Herodotus visited Microsoft and wrote down his experiences. It has that same sort of conversational tone, insightful on-the-ground observations, and facts no outsider might ever believe. Much of the article is a play-by-play account of the cat and mouse game David plays changing Messenger to track AOL's Instant Messenger protocol changes. AOL repeatedly tried to make it so M
5 0.79487395 767 high scalability-2010-01-27-Hot Scalability Links for January 28 2010
Introduction: Google's Research Areas of Interest: Building scalable, robust cluster applications . At Google we see distributed systems as a technology in its infancy, with huge gaps in the supporting research that represent some of the most important problems in the space. Here are some examples: Resource sharing, Balancing cost, performance, and reliability, Self-maintaining systems. Amazon SimpleDB: A Simple Way to Store Complex Data by Paul Tremblett. The most effective way I have found to understand SimpleDB is to think about it in terms of something else we all use and understand -- a spreadsheet. Rackspace Cloud Servers versus Amazon EC2: Performance Analysis . The Bitsource conducted a review of the two cloud computing platforms, Rackspace Cloud Servers and Amazon Elastic Compute Cloud (EC2), to get a general idea of overall system performance. Private Clouds Are Not The Future by Jame Hamilton. Private clouds are better than nothing but an investment in
6 0.79114527 1585 high scalability-2014-01-24-Stuff The Internet Says On Scalability For January 24th, 2014
8 0.78667814 1631 high scalability-2014-04-14-How do you even do anything without using EBS?
9 0.78048575 430 high scalability-2008-10-26-Should you use a SAN to scale your architecture?
10 0.77963924 1066 high scalability-2011-06-22-It's the Fraking IOPS - 1 SSD is 44,000 IOPS, Hard Drive is 180
11 0.77311265 1353 high scalability-2012-11-01-Cost Analysis: TripAdvisor and Pinterest costs on the AWS cloud
12 0.77105612 792 high scalability-2010-03-10-How FarmVille Scales - The Follow-up
13 0.76682442 142 high scalability-2007-11-05-Strategy: Diagonal Scaling - Don't Forget to Scale Out AND Up
14 0.76394039 1331 high scalability-2012-10-02-An Epic TripAdvisor Update: Why Not Run on the Cloud? The Grand Experiment.
15 0.76136523 584 high scalability-2009-04-27-Some Questions from a newbie
16 0.75932992 1299 high scalability-2012-08-06-Paper: High-Performance Concurrency Control Mechanisms for Main-Memory Databases
17 0.75397873 20 high scalability-2007-07-16-Paper: The Clustered Storage Revolution
18 0.74842697 1546 high scalability-2013-11-11-Ask HS: What is a good OLAP database choice with node.js?
19 0.7462796 689 high scalability-2009-08-28-Strategy: Solve Only 80 Percent of the Problem
20 0.74279958 1521 high scalability-2013-09-23-Salesforce Architecture - How they Handle 1.3 Billion Transactions a Day