high_scalability high_scalability-2009 high_scalability-2009-726 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Stanford Info Lab is taking pains to document a direction we've been moving for a while now, using RAM not just as a cache, but as the primary storage medium. Many quality products have built on this model. Even if the vision isn't radical, the paper does produce a lot of data backing up the transition, which is in itself helpful. From the The Abstract: Disk-oriented approaches to online storage are becoming increasingly problematic: they do not scale grace-fully to meet the needs of large-scale Web applications, and improvements in disk capacity have far out-stripped improvements in access latency and bandwidth. This paper argues for a new approach to datacenter storage called RAMCloud, where information is kept entirely in DRAM and large-scale systems are created by aggregating the main memories of thousands of commodity servers. We believe that RAMClouds can provide durable and available storage with 100-1000x the throughput of disk-based systems and 100-1000x lower access lat
sentIndex sentText sentNum sentScore
1 Stanford Info Lab is taking pains to document a direction we've been moving for a while now, using RAM not just as a cache, but as the primary storage medium. [sent-1, score-0.481]
2 Even if the vision isn't radical, the paper does produce a lot of data backing up the transition, which is in itself helpful. [sent-3, score-0.441]
3 From the The Abstract: Disk-oriented approaches to online storage are becoming increasingly problematic: they do not scale grace-fully to meet the needs of large-scale Web applications, and improvements in disk capacity have far out-stripped improvements in access latency and bandwidth. [sent-4, score-1.028]
4 This paper argues for a new approach to datacenter storage called RAMCloud, where information is kept entirely in DRAM and large-scale systems are created by aggregating the main memories of thousands of commodity servers. [sent-5, score-1.012]
5 We believe that RAMClouds can provide durable and available storage with 100-1000x the throughput of disk-based systems and 100-1000x lower access latency. [sent-6, score-0.43]
6 The combination of low latency and large scale will enable a new breed of data-intensive applications. [sent-7, score-0.309]
wordName wordTfidf (topN-words)
[('stanford', 0.254), ('info', 0.214), ('hpts', 0.199), ('memoryby', 0.199), ('ousterhout', 0.199), ('shoup', 0.187), ('ramcloud', 0.187), ('ramclouds', 0.187), ('improvements', 0.185), ('randy', 0.162), ('articleshacker', 0.162), ('pains', 0.155), ('problematic', 0.146), ('aggregating', 0.146), ('lab', 0.138), ('breed', 0.138), ('argues', 0.138), ('durable', 0.138), ('memories', 0.134), ('storage', 0.131), ('paper', 0.124), ('backing', 0.122), ('dram', 0.122), ('linden', 0.12), ('radical', 0.118), ('hamilton', 0.109), ('direction', 0.108), ('entirely', 0.103), ('increasingly', 0.103), ('choosing', 0.102), ('transition', 0.102), ('greg', 0.099), ('produce', 0.098), ('abstract', 0.097), ('vision', 0.097), ('james', 0.097), ('john', 0.095), ('latency', 0.09), ('document', 0.087), ('becoming', 0.086), ('access', 0.086), ('approaches', 0.086), ('commodity', 0.084), ('ssd', 0.084), ('enable', 0.081), ('kept', 0.077), ('understanding', 0.077), ('meet', 0.076), ('datacenter', 0.075), ('believe', 0.075)]
simIndex simValue blogId blogTitle
same-blog 1 1.0000001 726 high scalability-2009-10-22-Paper: The Case for RAMClouds: Scalable High-Performance Storage Entirely in DRAM
Introduction: Stanford Info Lab is taking pains to document a direction we've been moving for a while now, using RAM not just as a cache, but as the primary storage medium. Many quality products have built on this model. Even if the vision isn't radical, the paper does produce a lot of data backing up the transition, which is in itself helpful. From the The Abstract: Disk-oriented approaches to online storage are becoming increasingly problematic: they do not scale grace-fully to meet the needs of large-scale Web applications, and improvements in disk capacity have far out-stripped improvements in access latency and bandwidth. This paper argues for a new approach to datacenter storage called RAMCloud, where information is kept entirely in DRAM and large-scale systems are created by aggregating the main memories of thousands of commodity servers. We believe that RAMClouds can provide durable and available storage with 100-1000x the throughput of disk-based systems and 100-1000x lower access lat
Introduction: Most every programmer who gets sucked into deep performance analysis for long running processes eventually realizes memory allocation is the heart of evil at the center of many of their problems. So you replace malloc with something less worse. Or you tune your garbage collector like a fine ukulele. But there's a smarter approach brought to you from the folks at RAMCloud , a Stanford University production, which is a large scale, distributed, in-memory key-value database. What they've found is that typical memory management approaches don't work and using a log structured approach yields massive benefits: Performance measurements of log-structured memory in RAMCloud show that it enables high client through- put at 80-90% memory utilization, even with artificially stressful workloads. In the most stressful workload, a single RAMCloud server can support 270,000-410,000 durable 100-byte writes per second at 90% memory utilization. The two-level approach to cleaning improves perform
3 0.11801325 742 high scalability-2009-11-17-10 eBay Secrets for Planet Wide Scaling
Introduction: You don't even have to make a bid, Randy Shoup, an eBay Distinguished Architect, gives this presentation on how eBay scales, for free. Randy has done a fabulous job in this presentation and in other talks listed at the end of this post getting at the heart of the principles behind scalability. It's more about ideas of how things work and fit together than a focusing on a particular technology stack. Impressive Stats In case you weren't sure, eBay is big, with lots of: users, data, features, and change... Over 89 million active users worldwide 190 million items for sale in 50,000 categories Over 8 billion URL requests per day Hundreds of new features per quarter Roughly 10% of items are listed or ended every day In 39 countries and 10 languages 24x7x365 70 billion read / write operations / day Processes 50TB of new, incremental data per day Analyzes 50PB of data per day 10 Lessons The presentation does a good job explaining each lesson, but the list is.
4 0.11196794 538 high scalability-2009-03-16-Are Cloud Based Memory Architectures the Next Big Thing?
Introduction: We are on the edge of two potent technological changes: Clouds and Memory Based Architectures. This evolution will rip open a chasm where new players can enter and prosper. Google is the master of disk. You can't beat them at a game they perfected. Disk based databases like SimpleDB and BigTable are complicated beasts, typical last gasp products of any aging technology before a change. The next era is the age of Memory and Cloud which will allow for new players to succeed. The tipping point will be soon. Let's take a short trip down web architecture lane: It's 1993: Yahoo runs on FreeBSD, Apache, Perl scripts and a SQL database It's 1995: Scale-up the database. It's 1998: LAMP It's 1999: Stateless + Load Balanced + Database + SAN It's 2001: In-memory data-grid. It's 2003: Add a caching layer. It's 2004: Add scale-out and partitioning. It's 2005: Add asynchronous job scheduling and maybe a distributed file system. It's 2007: Move it all into the cloud. It's 2008: C
5 0.10207167 661 high scalability-2009-07-25-Latency is Everywhere and it Costs You Sales - How to Crush it
Introduction: Update 8 : The Cost of Latency by James Hamilton. James summarizing some latency info from Steve Souder , Greg Linden , and Marissa Mayer . Speed [is] an undervalued and under-discussed asset on the web. Update 7: How do you know when you need more memcache servers? . Dathan Pattishall talks about using memcache not to scale, but to reduce latency and reduce I/O spikes, and how to use stats to know when more servers are needed. Update 6: Stock Traders Find Speed Pays, in Milliseconds . Goldman Sachs is making record profits off a 500 millisecond trading advantage. Yes, latency matters. As an interesting aside, Libet found 500 msecs is about the time it takes the brain to weave together an experience of consciousness from all our sensor inputs. Update 5: Shopzilla's Site Redo - You Get What You Measure . At the Velocity conference Phil Dixon, from Shopzilla, presented data showing a 5 second speed up resulted in a 25% increase in page views, a 10% increas
6 0.099799871 614 high scalability-2009-06-01-Guess How Many Users it Takes to Kill Your Site?
7 0.092746831 473 high scalability-2008-12-20-Second Life Architecture - The Grid
8 0.087139428 734 high scalability-2009-10-30-Hot Scalabilty Links for October 30 2009
10 0.085941941 750 high scalability-2009-12-16-Building Super Scalable Systems: Blade Runner Meets Autonomic Computing in the Ambient Cloud
11 0.080776393 1291 high scalability-2012-07-25-Vertical Scaling Ascendant - How are SSDs Changing Architectures?
12 0.079646558 980 high scalability-2011-01-28-Stuff The Internet Says On Scalability For January 28, 2011
13 0.076624937 1346 high scalability-2012-10-24-Saving Cash Using Less Cache - 90% Savings in the Caching Tier
14 0.076308832 1118 high scalability-2011-09-19-Big Iron Returns with BigMemory
15 0.075140819 284 high scalability-2008-03-19-RAD Lab is Creating a Datacenter Operating System
16 0.074720621 448 high scalability-2008-11-22-Google Architecture
17 0.074711636 1369 high scalability-2012-12-10-Switch your databases to Flash storage. Now. Or you're doing it wrong.
18 0.073914431 580 high scalability-2009-04-24-INFOSCALE 2009 in June in Hong Kong
19 0.073721543 288 high scalability-2008-03-25-Paper: On Designing and Deploying Internet-Scale Services
20 0.07308875 1052 high scalability-2011-06-03-Stuff The Internet Says On Scalability For June 3, 2011
topicId topicWeight
[(0, 0.128), (1, 0.047), (2, 0.035), (3, 0.045), (4, -0.047), (5, 0.049), (6, 0.011), (7, 0.002), (8, -0.034), (9, 0.024), (10, -0.001), (11, -0.058), (12, -0.024), (13, 0.055), (14, -0.022), (15, 0.047), (16, 0.01), (17, -0.014), (18, -0.012), (19, 0.004), (20, 0.007), (21, 0.048), (22, -0.038), (23, 0.034), (24, -0.027), (25, -0.005), (26, -0.019), (27, -0.032), (28, -0.044), (29, -0.043), (30, 0.036), (31, -0.006), (32, 0.016), (33, -0.046), (34, -0.007), (35, 0.016), (36, -0.009), (37, -0.015), (38, 0.007), (39, 0.027), (40, 0.03), (41, -0.033), (42, 0.034), (43, 0.027), (44, -0.003), (45, -0.013), (46, -0.048), (47, -0.05), (48, -0.012), (49, -0.053)]
simIndex simValue blogId blogTitle
same-blog 1 0.96465814 726 high scalability-2009-10-22-Paper: The Case for RAMClouds: Scalable High-Performance Storage Entirely in DRAM
Introduction: Stanford Info Lab is taking pains to document a direction we've been moving for a while now, using RAM not just as a cache, but as the primary storage medium. Many quality products have built on this model. Even if the vision isn't radical, the paper does produce a lot of data backing up the transition, which is in itself helpful. From the The Abstract: Disk-oriented approaches to online storage are becoming increasingly problematic: they do not scale grace-fully to meet the needs of large-scale Web applications, and improvements in disk capacity have far out-stripped improvements in access latency and bandwidth. This paper argues for a new approach to datacenter storage called RAMCloud, where information is kept entirely in DRAM and large-scale systems are created by aggregating the main memories of thousands of commodity servers. We believe that RAMClouds can provide durable and available storage with 100-1000x the throughput of disk-based systems and 100-1000x lower access lat
2 0.72024846 1369 high scalability-2012-12-10-Switch your databases to Flash storage. Now. Or you're doing it wrong.
Introduction: This is a guest post by Brian Bulkowski , CTO and co-founder of Aerospike , a leading clustered NoSQL database, has worked in the area of high performance commodity systems since 1989. Why flash rules for databases The economics of flash memory are staggering. If you’re not using SSD, you are doing it wrong. Not quite true, but close. Some small applications fit entirely in memory – less than 100GB – great for in-memory solutions. There’s a place for rotational drives (HDD) in massive streaming analytics and petabytes of data. But for the vast space between, flash has become the only sensible option. For example, the Samsung 840 costs $180 for 250GB. The speed rating for this drive is rated by the manufacturer at 96,000 random 4K read IOPS, and 61,000 random 4K write IOPS. The Samsung 840 is not alone at this price performance. A 300GB Intel 320 is $450. An OCZ Vertex 4 256GB is $235, with the Intel being rated as slowest, but our internal testing showing
Introduction: “ Data is everywhere, never be at a single location. Not scalable, not maintainable. ” –Alex Szalay While Galileo played life and death doctrinal games over the mysteries revealed by the telescope, another revolution went unnoticed, the microscope gave up mystery after mystery and nobody yet understood how subversive would be what it revealed. For the first time these new tools of perceptual augmentation allowed humans to peek behind the veil of appearance. A new new eye driving human invention and discovery for hundreds of years. Data is another material that hides, revealing itself only when we look at different scales and investigate its underlying patterns. If the universe is truly made of information , then we are looking into truly primal stuff. A new eye is needed for Data and an ambitious project called Data-scope aims to be the lens. A detailed paper on the Data-Scope tells more about what it is: The Data-Scope is a new scientific instrum
4 0.69740671 967 high scalability-2011-01-03-Stuff The Internet Says On Scalability For January 3, 2010
Introduction: Submitted for your reading pleasure... Quotable Quotes @hofmanndavid : Performance and scalability anxiety makes developers want to catch the flying butterflies @tivrfoa : "Scalability solutions aren't magic. They involve partitioning, indexing and replication." Twitter engineer Alan Perlis: Fools ignore complexity; pragmatists suffer it; experts avoid it; geniuses remove it. CIO update: Post-mortem on the Skype outage . Interesting tale of a cascading collapse in complex, distributed, interactive systems. For more background see the highly illuminating Explaining Supernodes by Dan York. RethinkDB and SSD Databases. SSD was not a revolution by Kevin Burton. What’s really shocking to me, is that while SSD and flash storage is very exciting, it wasn’t as revolutionary in 2010 as I would have liked to have seen. The case for Datastore-Side-Scripting . Russell Sullivan predicts real-time web applications are going in the direction of being enti
5 0.69520909 1316 high scalability-2012-09-04-Changing Architectures: New Datacenter Networks Will Set Your Code and Data Free
Introduction: One consequence of IT standardization and commodification has been Google’s datacenter is the computer view of the world. In that view all compute resources (memory, CPU, storage) are fungible. They are interchangeable and location independent, individual computers lose identity and become just a part of a service. Thwarting that nirvana has been the abysmal performance of commodity datacenter networks which have caused the preference of architectures that favor the collocation of state and behaviour on the same box. MapReduce famously ships code over to storage nodes for just this reason. Change the network and you change the fundamental assumption driving collocation based software architectures. You are then free to store data anywhere and move compute anywhere you wish. The datacenter becomes the computer. On the host side with an x8 slot running at PCI-Express 3.0 speeds able to push 8GB/sec (that’s bytes) of bandwidth in both directions, we have
6 0.68460679 1177 high scalability-2012-01-19-Is it time to get rid of the Linux OS model in the cloud?
7 0.68366611 1338 high scalability-2012-10-11-RAMCube: Exploiting Network Proximity for RAM-Based Key-Value Store
9 0.68104482 1511 high scalability-2013-09-04-Wide Fast SATA: the Recipe for Hot Performance
10 0.66884542 104 high scalability-2007-10-01-SmugMug Found their Perfect Storage Array
11 0.65401721 1407 high scalability-2013-02-15-Stuff The Internet Says On Scalability For February 15, 2013
12 0.64568132 368 high scalability-2008-08-17-Wuala - P2P Online Storage Cloud
13 0.64163691 423 high scalability-2008-10-19-Alternatives to Google App Engine
14 0.63884628 1327 high scalability-2012-09-21-Stuff The Internet Says On Scalability For September 21, 2012
15 0.63681805 784 high scalability-2010-02-25-Paper: High Performance Scalable Data Stores
16 0.63275397 463 high scalability-2008-12-09-Rules of Thumb in Data Engineering
17 0.63131297 980 high scalability-2011-01-28-Stuff The Internet Says On Scalability For January 28, 2011
18 0.63093489 1637 high scalability-2014-04-25-Stuff The Internet Says On Scalability For April 25th, 2014
19 0.62912101 1270 high scalability-2012-06-22-Stuff The Internet Says On Scalability For June 22, 2012
20 0.62856781 823 high scalability-2010-05-05-How will memristors change everything?
topicId topicWeight
[(1, 0.158), (2, 0.157), (43, 0.3), (61, 0.171), (77, 0.035), (79, 0.078)]
simIndex simValue blogId blogTitle
1 0.90324539 505 high scalability-2009-02-01-More Chips Means Less Salsa
Introduction: Yes, I just got through watching the Superbowl so chips and salsa are on my mind and in my stomach. In recreational eating more chips requires downing more salsa. With mulitcore chips it turns out as cores go up salsa goes down, salsa obviously being a metaphor for speed. Sandia National Laboratories found in their simulations: a significant increase in speed going from two to four multicores, but an insignificant increase from four to eight multicores. Exceeding eight multicores causes a decrease in speed. Sixteen multicores perform barely as well as two, and after that, a steep decline is registered as more cores are added. The problem is the lack of memory bandwidth as well as contention between processors over the memory bus available to each processor. The implication for those following a diagonal scaling strategy is to work like heck to make your system fit within eight multicores. After that you'll need to consider some sort of partitioning strategy. What's interesti
same-blog 2 0.88281173 726 high scalability-2009-10-22-Paper: The Case for RAMClouds: Scalable High-Performance Storage Entirely in DRAM
Introduction: Stanford Info Lab is taking pains to document a direction we've been moving for a while now, using RAM not just as a cache, but as the primary storage medium. Many quality products have built on this model. Even if the vision isn't radical, the paper does produce a lot of data backing up the transition, which is in itself helpful. From the The Abstract: Disk-oriented approaches to online storage are becoming increasingly problematic: they do not scale grace-fully to meet the needs of large-scale Web applications, and improvements in disk capacity have far out-stripped improvements in access latency and bandwidth. This paper argues for a new approach to datacenter storage called RAMCloud, where information is kept entirely in DRAM and large-scale systems are created by aggregating the main memories of thousands of commodity servers. We believe that RAMClouds can provide durable and available storage with 100-1000x the throughput of disk-based systems and 100-1000x lower access lat
3 0.84849817 893 high scalability-2010-09-03-Hot Scalability Links For Sep 3, 2010
Introduction: With summer almost gone, it's time to fall into some good links... Hibari - distributed, fault tolerant, highly available key-value store written in Erlang. In this video Scott Lystig Fritchie gives a very good overview of the newest key-value store. Tweets of Gold lenidot : with 12 staff, @ tumblr serves 1.5billion pageviews/month and 25,000 signups/day. Now that's scalability! jmtan24 : Funny that whenever a high scalability article comes out, it always mention the shared nothing approach mfeathers : When life gives you lemons, you can have decades-long conquest to convert lemons to oranges, or you can make lemonade. OyvindIsene : Met an old man with mustache today, he had no opinion on #noSQL . Note to myself: Don't grow a mustache, now or later. vlad003 : Isn't it interesting how P2P distributes data while Cloud Computing centralizes it? And they're both said to be the future. You may be interested in a new DevOps Meetup organized by Dave
4 0.82180697 1182 high scalability-2012-01-27-Stuff The Internet Says On Scalability For January 27, 2012
Introduction: If you’ve got the time, we’ve got the HighScalability: 9nm : IBM's carbon nanotube transistor that outperforms silicon; YouTube : 4 Billion Views/Day; 864GB RAM : 37signals Memcache, $12K Quotable Quotes: Chad Dickerson : You can only get growth by feeding opportunities. @launchany : It amazes me how many NoSQL database vendors spend more time detailing their scalability and no time detailing the data model and design Google : Let's make TCP faster. WhatsApp : we are now able to easily push our systems to over 2 million tcp connections! Sidney Dekker : In a complex system…doing the same thing twice will not predictably or necessarily lead to the same results. @Rasmusfjord : Just heard about an Umbraco site running on Azure that handles 20.000 requests /*second* Herb Sutter with an epic post, Welcome to the Jungle , touching on a lot of themes we've explored on HighScalability, only in a dramatically more competent way. What's after
5 0.81142592 1336 high scalability-2012-10-09-Batoo JPA - The new JPA Implementation that runs over 15 times faster...
Introduction: This post is by Hasan Ceylan , an Open Source software enthusiast from Istanbul. I loved the JPA 1.0 back in early 2000s. I started using it together with EJB 3.0 even before the stable releases. I loved it so much that I contributed bits and parts for JBoss 3.x implementations. Those were the days our company was considerably still small in size. Creating new features and applications were more priority than the performance, because there were a lot of ideas that we have and we needed to develop and market those as fast as we can. Now, we no longer needed to write tedious and error prone xml descriptions for the data model and deployment descriptors. Nor we needed to use the curse called “XDoclet”. On the other side, our company grew steadily, our web site has become the top portal in the country for live events and ticketing. We now had the performance problems! Although the company grew considerably, due to the economics in the industry, we did not make a lot of money. The ch
6 0.79897296 1624 high scalability-2014-04-01-The Mullet Cloud Selection Pattern
7 0.79403764 37 high scalability-2007-07-28-Product: Web Log Storming
8 0.74097359 470 high scalability-2008-12-18-Risk Analysis on the Cloud (Using Excel and GigaSpaces)
9 0.74011499 1088 high scalability-2011-07-27-Making Hadoop 1000x Faster for Graph Problems
10 0.71798551 1603 high scalability-2014-02-28-Stuff The Internet Says On Scalability For February 28th, 2014
11 0.71233755 937 high scalability-2010-11-09-Paper: Hyder - Scaling Out without Partitioning
12 0.70522255 720 high scalability-2009-10-12-High Performance at Massive Scale – Lessons learned at Facebook
13 0.69370836 1399 high scalability-2013-02-05-Ask HighScalability: Memcached and Relations
14 0.6903742 36 high scalability-2007-07-28-Product: Web Log Expert
15 0.68880159 1123 high scalability-2011-09-23-The Real News is Not that Facebook Serves Up 1 Trillion Pages a Month…
16 0.68578112 1131 high scalability-2011-10-24-StackExchange Architecture Updates - Running Smoothly, Amazon 4x More Expensive
17 0.68261969 342 high scalability-2008-06-08-Search fast in million rows
18 0.68071115 810 high scalability-2010-04-14-Parallel Information Retrieval and Other Search Engine Goodness
19 0.67924577 931 high scalability-2010-10-28-Notes from A NOSQL Evening in Palo Alto
20 0.67906654 1031 high scalability-2011-04-28-PaaS on OpenStack - Run Applications on Any Cloud, Any Time Using Any Thing