high_scalability high_scalability-2009 high_scalability-2009-584 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Hello highscalability world. I just discovered this site yesterday in a search for a scalability resource and was very pleased to find such useful information. I have some questions regarding distributed caching that I was hoping the scalability intelligentsia trafficking this forum could answer. I apologize for my lack of technical knowledge; I'm hoping this site will increase said knowledge! Feel free to answer all or as much as you want. Thank you in advance for your responses and thank you for a great resource! 1.) What are the standard benchmarks used to measure the performance of memcached or mySQL/memcached working together (from web 2.0 companies etc)? 2.) The little research I've conducted on this site suggests that most web 2.0 companies use a combination of mySQL and a hacked memcached (and potentially sharding). Does anyone know if any of these companies use an enterprise vendor for their distributed caching layer? (At this point in time I've only heard of Jive soft
sentIndex sentText sentNum sentScore
1 I just discovered this site yesterday in a search for a scalability resource and was very pleased to find such useful information. [sent-2, score-0.586]
2 I have some questions regarding distributed caching that I was hoping the scalability intelligentsia trafficking this forum could answer. [sent-3, score-0.643]
3 I apologize for my lack of technical knowledge; I'm hoping this site will increase said knowledge! [sent-4, score-0.865]
4 Thank you in advance for your responses and thank you for a great resource! [sent-6, score-0.548]
5 ) What are the standard benchmarks used to measure the performance of memcached or mySQL/memcached working together (from web 2. [sent-8, score-0.396]
6 ) The little research I've conducted on this site suggests that most web 2. [sent-11, score-0.547]
7 0 companies use a combination of mySQL and a hacked memcached (and potentially sharding). [sent-12, score-0.543]
8 Does anyone know if any of these companies use an enterprise vendor for their distributed caching layer? [sent-13, score-0.439]
9 (At this point in time I've only heard of Jive software using Coherence). [sent-14, score-0.102]
10 0 oriented startup, what are the database/distributed caching requirements typically needed to get off the ground and grow at a fairly rapid pace? [sent-17, score-0.61]
11 0 industry (facebook, twitter, myspace, PoF, Flickr etc, I'm ignoring google/amazon here because they have a proprietary caching layer) what is the most common, scalable back-end setup (mySQL/memcached/sharding etc)? [sent-20, score-0.585]
12 What features does said setup lack that it really needs? [sent-22, score-0.47]
wordName wordTfidf (topN-words)
[('thank', 0.33), ('hoping', 0.228), ('etc', 0.205), ('apologize', 0.194), ('hacked', 0.182), ('caching', 0.18), ('pof', 0.174), ('lack', 0.172), ('companies', 0.158), ('setup', 0.149), ('said', 0.149), ('ignoring', 0.148), ('knowledge', 0.145), ('conducted', 0.14), ('hello', 0.138), ('yesterday', 0.132), ('pace', 0.128), ('myspace', 0.125), ('regarding', 0.125), ('layer', 0.123), ('site', 0.122), ('benchmarks', 0.117), ('suggests', 0.115), ('advance', 0.114), ('pleased', 0.114), ('coherence', 0.114), ('resource', 0.112), ('memcached', 0.111), ('forum', 0.11), ('proprietary', 0.108), ('discovered', 0.106), ('players', 0.105), ('responses', 0.104), ('heard', 0.102), ('flickr', 0.102), ('vendor', 0.101), ('web', 0.095), ('potentially', 0.092), ('ground', 0.092), ('oriented', 0.09), ('typically', 0.084), ('fairly', 0.084), ('insight', 0.082), ('rapid', 0.08), ('research', 0.075), ('sharding', 0.075), ('terms', 0.074), ('feel', 0.074), ('measure', 0.073), ('startup', 0.073)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999994 584 high scalability-2009-04-27-Some Questions from a newbie
Introduction: Hello highscalability world. I just discovered this site yesterday in a search for a scalability resource and was very pleased to find such useful information. I have some questions regarding distributed caching that I was hoping the scalability intelligentsia trafficking this forum could answer. I apologize for my lack of technical knowledge; I'm hoping this site will increase said knowledge! Feel free to answer all or as much as you want. Thank you in advance for your responses and thank you for a great resource! 1.) What are the standard benchmarks used to measure the performance of memcached or mySQL/memcached working together (from web 2.0 companies etc)? 2.) The little research I've conducted on this site suggests that most web 2.0 companies use a combination of mySQL and a hacked memcached (and potentially sharding). Does anyone know if any of these companies use an enterprise vendor for their distributed caching layer? (At this point in time I've only heard of Jive soft
2 0.17666055 1011 high scalability-2011-03-25-Did the Microsoft Stack Kill MySpace?
Introduction: Robert Scoble wrote a fascinating case study, MySpace’s death spiral: insiders say it’s due to bets on Los Angeles and Microsoft , where he reports MySpace insiders blame the Microsoft stack on why they lost the great social network race to Facebook. Does anyone know if this is true? What's the real story? I was wondering because it doesn't seem to track with the MySpace Architecture post that I did in 2009, where they seem happy with their choices and had stats to back up their improvements. Why this matters is it's a fascinating model for startups to learn from. What does it really take to succeed? Is it the people or the stack? Is it the organization or the technology? Is it the process or the competition? Is the quality of the site or the love of the users? So much to consider and learn from. Some conjectures from the article: Myspace didn't have programming talent capable of scaling the site to compete with Facebook. Choosing the Microsoft stack made it difficul
3 0.12494518 1014 high scalability-2011-03-31-8 Lessons We Can Learn from the MySpace Incident - Balance, Vision, Fearlessness
Introduction: A surprising amount of heat and light was generated by the whole Micrsoft vs MySpace discussion. Why people feel so passionate about this I'm not quite sure, but fortunately for us, in the best sense of the web, it generated an amazing number of insightful comments and observations. If we stand back and take a look at the whole incident, what can we take a way that might help us in the future? All computer companies are technology companies first. A repeated theme was that you can't be an entertainment company first. You are a technology company providing entertainment using technology. The tech can inform the entertainment side, the entertainment side drives features, but they really can't be separated. An awesome stack that does nothing is useless. A great idea on a poor stack is just as useless. There's a difficult balance that must be achieved and both management and developers must be aware that there's something to balance. All pigs are equal . All business f
4 0.10732894 360 high scalability-2008-08-04-A Bunch of Great Strategies for Using Memcached and MySQL Better Together
Introduction: The primero recommendation for speeding up a website is almost always to add cache and more cache. And after that add a little more cache just in case. Memcached is almost always given as the recommended cache to use. What we don't often hear is how to effectively use a cache in our own products. MySQL hosted two excellent webinars (referenced below) on the subject of how to deploy and use memcached. The star of the show, other than MySQL of course, is Farhan Mashraqi of Fotolog. You may recall we did an earlier article on Fotolog in Secrets to Fotolog's Scaling Success , which was one of my personal favorites. Fotolog, as they themselves point out, is probably the largest site nobody has ever heard of, pulling in more page views than even Flickr. Fotolog has 51 instances of memcached on 21 servers with 175G in use and 254G available. As a large successful photo-blogging site they have very demanding performance and scaling requirements. To meet those requirements they've developed a
5 0.10502602 638 high scalability-2009-06-26-PlentyOfFish Architecture
Introduction: Update 5 : PlentyOfFish Update - 6 Billion Pageviews And 32 Billion Images A Month Update 4 : Jeff Atwood costs out Markus' scale up approach against a scale out approach and finds scale up wanting. The discussion in the comments is as interesting as the article. My guess is Markus doesn't want to rewrite his software to work across a scale out cluster so even if it's more expensive scale up works better for his needs. Update 3 : POF now has 200 million images and serves 10,000 images served per second. They'll be moving to a 250,000 IOPS RamSan to handle the load. Also upgraded to a core database machine with 512 GB of RAM, 32 CPU’s, SQLServer 2008 and Windows 2008. Update 2 : This seems to be a POF Peer1 love fest infomercial . It's pretty content free, but the production values are high. Lots of quirky sounds and fish swimming on the screen. Update : by Facebook standards Read/WriteWeb says POF is worth a cool one billion dollars . It helps to talk like Dr. Evil whe
6 0.10451087 1361 high scalability-2012-11-22-Gone Fishin': PlentyOfFish Architecture
7 0.10445695 226 high scalability-2008-01-28-DR-BC for web-DB servers
8 0.10413797 632 high scalability-2009-06-15-starting small with growth in mind
9 0.098690391 788 high scalability-2010-03-04-How MySpace Tested Their Live Site with 1 Million Concurrent Users
10 0.097203791 416 high scalability-2008-10-15-Oracle opens Coherence Incubator
11 0.095476031 194 high scalability-2007-12-26-Golden rule of web caching
13 0.093149543 573 high scalability-2009-04-16-Serving 250M quotes-day at CNBC.com with aiCache
14 0.091657601 1 high scalability-2007-07-06-Start Here
15 0.090704717 259 high scalability-2008-02-25-Any Suggestions for the Architecture Template?
16 0.090704717 260 high scalability-2008-02-25-Architecture Template Advice Needed
18 0.089857027 313 high scalability-2008-05-02-Friends for Sale Architecture - A 300 Million Page View-Month Facebook RoR App
19 0.08936812 863 high scalability-2010-07-22-How can we spark the movement of research out of the Ivory Tower and into production?
20 0.08899951 511 high scalability-2009-02-12-MySpace Architecture
topicId topicWeight
[(0, 0.157), (1, 0.022), (2, 0.009), (3, -0.08), (4, 0.042), (5, -0.037), (6, -0.102), (7, -0.025), (8, 0.014), (9, 0.063), (10, -0.042), (11, -0.01), (12, -0.022), (13, 0.06), (14, -0.015), (15, -0.054), (16, 0.039), (17, -0.044), (18, 0.022), (19, 0.032), (20, 0.006), (21, 0.012), (22, -0.004), (23, 0.024), (24, -0.049), (25, -0.045), (26, 0.024), (27, 0.017), (28, 0.019), (29, 0.013), (30, -0.003), (31, 0.046), (32, 0.009), (33, -0.056), (34, 0.018), (35, -0.014), (36, 0.006), (37, 0.012), (38, 0.006), (39, 0.027), (40, -0.012), (41, 0.035), (42, 0.043), (43, -0.024), (44, -0.042), (45, -0.003), (46, -0.037), (47, 0.02), (48, 0.028), (49, 0.023)]
simIndex simValue blogId blogTitle
same-blog 1 0.96616381 584 high scalability-2009-04-27-Some Questions from a newbie
Introduction: Hello highscalability world. I just discovered this site yesterday in a search for a scalability resource and was very pleased to find such useful information. I have some questions regarding distributed caching that I was hoping the scalability intelligentsia trafficking this forum could answer. I apologize for my lack of technical knowledge; I'm hoping this site will increase said knowledge! Feel free to answer all or as much as you want. Thank you in advance for your responses and thank you for a great resource! 1.) What are the standard benchmarks used to measure the performance of memcached or mySQL/memcached working together (from web 2.0 companies etc)? 2.) The little research I've conducted on this site suggests that most web 2.0 companies use a combination of mySQL and a hacked memcached (and potentially sharding). Does anyone know if any of these companies use an enterprise vendor for their distributed caching layer? (At this point in time I've only heard of Jive soft
2 0.78197724 1 high scalability-2007-07-06-Start Here
Introduction: This page is here to help you get started using High Scalability. Here are a few useful topics to get you going... Why does the High Scalability site exist? Good things to read. Participate by adding your own links to interesting sites and articles. Participate by signing up for the RSS feed. Consider the many benefits of registering as a user. How do I get notification of content and comment changes? Contact High Scalability. About. Why does the High Scalability site exist? To help you build successful scalable websites. This site tries to bring together all the lore, art, science, practice, and experience of building scalable websites into one place so you can learn how to build your website with confidence. When it becomes clear you must grow your website or die, most people have no idea where to start. It's not a skill you learn in school or pick up from a magazine article on a plane flight home. No, building scalable systems is a body o
3 0.72145158 106 high scalability-2007-10-02-Secrets to Fotolog's Scaling Success
Introduction: Fotolog, a social blogging site centered around photos, grew from about 300 thousand users in 2004 to over 11 million users in 2007. Though they initially experienced the inevitable pains of rapid growth, they overcame their problems and now manage over 300 million photos and 800,000 new photos are added each day. Generating all that fabulous content are 20 million unique monthly visitors and a volunteer army of 30,000 new users each day. They did so well a very impressed suitor bought them out for a cool $90 million. That's scale meets success by anyone standards. How did they do it? Site: http://www.fotolog.com Information Sources Scaling the World's Largest Photo Blogging Community Congrats to Fotolog on $90mm sale to Hi-Media Fotolog overtaking Flickr? Fotolog Hits 11 Million Members and 300 Million Photos Posted Site of the Week: Fotolog.com by PC Magazine CEO John Borthwick's Blog . DBA Frank Mash's Blog Fotolog, lessons learnt by John B
4 0.71157998 232 high scalability-2008-01-29-When things aren't scalable
Introduction: OK, I know this site is for scalable web site design. But as there aren't any sites I can find for graceful failure under "slashdotted" like pressure I'll ask here. Does anyone have a sensible way, once you have a "web application" that either won't scale, or can't scale, that you can give some users a good consistent experience and bounce other users to a busy site page. I have seen sites do this to varying degrees, some of which work better than others, but no explanations beyond simply bouncing requests to a "we're busy page server" when you have more than a given number of connections. This is obviously useless as a web page likely requires multiple connection (ignoring keep-alive, pipelining etc) multiple connection to completely render properly. The normal problem is users getting a page and not the "furniture" for that page like images or css. Other problems are having to wait ages to get the busy page or the site being slow even if you do "get in". And some site let
5 0.70558351 51 high scalability-2007-07-31-Book: Scalable Internet Architectures
Introduction: As a developer, you are aware of the increasing concern amongst developers and site architects that websites be able to handle the vast number of visitors that flood the Internet on a daily basis. Scalable Internet Architecture addresses these concerns by teaching you both good and bad design methodologies for building new sites and how to scale existing websites to robust, high-availability websites. Primarily example-based, the book discusses major topics in web architectural design, presenting existing solutions and how they work. Technology budget tight? This book will work for you, too, as it introduces new and innovative concepts to solving traditionally expensive problems without a large technology budget. Using open source and proprietary examples, you will be engaged in best practice design methodologies for building new sites, as well as appropriately scaling both growing and shrinking sites. Website development help has arrived in the form of Scalable Internet Architecture.
6 0.70163536 33 high scalability-2007-07-26-ThemBid Architecture
7 0.69458932 8 high scalability-2007-07-12-Should I use LAMP or Windows?
9 0.67728174 10 high scalability-2007-07-15-Book: Building Scalable Web Sites
10 0.67641532 711 high scalability-2009-09-22-How Ravelry Scales to 10 Million Requests Using Rails
11 0.67213207 1349 high scalability-2012-10-29-Gone Fishin': Welcome to High Scalability
12 0.67058152 121 high scalability-2007-10-14-Newbie in scalability design issues
13 0.67054403 206 high scalability-2008-01-10-MONO ASP.NET. Will it make the web???
14 0.66260827 375 high scalability-2008-09-01-A Scalability checklist?
15 0.65971535 167 high scalability-2007-11-27-Starting a website from scratch - what technologies should I use?
16 0.65767223 1011 high scalability-2011-03-25-Did the Microsoft Stack Kill MySpace?
17 0.65647918 632 high scalability-2009-06-15-starting small with growth in mind
18 0.65290403 276 high scalability-2008-03-15-New Website Design Considerations
19 0.65247071 2 high scalability-2007-07-08-Welcome to High Scalability
20 0.65144956 571 high scalability-2009-04-15-Using HTTP cache headers effectively
topicId topicWeight
[(1, 0.146), (2, 0.158), (10, 0.443), (61, 0.119), (79, 0.032)]
simIndex simValue blogId blogTitle
1 0.99264288 874 high scalability-2010-08-07-ArchCamp: Scalable Databases (NoSQL)
Introduction: ArchCamp: Scalable Databasess (NoSQL) The ArchCamp unconference was held this past Friday at HackerDojo in Mountain View, CA. There was plenty of pizza, beer, and great conversation. This session started out free-form, but shaped up pretty quickly into a discussion of the popular open source scalable NoSQL databases and the architectural categories in which they belong.
2 0.95556664 430 high scalability-2008-10-26-Should you use a SAN to scale your architecture?
Introduction: This is a question everyone must struggle with when building out their datacenter. Storage choices are always the ones I have the least confidence in. David Marks in his blog You Can Change It Later! asks the question Should I get a SAN to scale my site architecture? and answers no. A better solution is to use commodity hardware, directly attach storage on servers, and partition across servers to scale and for greater availability. David's reasoning is interesting: A SAN creates a SPOF (single point of failure) that is dependent on a vendor to fly and fix when there's a problem. This can lead to long down times during this outage you have no access to your data at all. Using easily available commodity hardware minimizes risks to your company, it's not just about saving money. Zooming over to Fry's to buy emergency equipment provides the kind of agility startups need in order to respond quickly to ever changing situations. It's hard to beat the power and flexibility (backup
3 0.95555234 178 high scalability-2007-12-10-1 Master, N Slaves
Introduction: Hello all, Reading the site you can note that "1 Master for writes, N Slaves for reads" scheme is used offen. How is this implemented? Who decides where writes and reads go? Something in application level or specific database proxies, like Slony-I? Thanks.
Introduction: This is a guest post by Ali Khajeh-Hosseini , Technical Lead at PlanForCloud . The original article was published on their site . With 29 cloud price reductions I thought it would be interesting to see how the bottom line would change compared to an article we published last year . The result is surprisingly little for TripAdvisor because prices for On Demand instances have not dropped as fast as for other other instances types. Over the last year and a half, we counted 29 price reductions in cloud services provided by AWS, Google Compute Engine, Windows Azure, and Rackspace Cloud. Price reductions have a direct effect on cloud users, but given the usual tiny reductions, how significant is that effect on the bottom line? Last year I wrote about cloud cost forecasts for TripAdvisor and Pinterest . TripAdvisor was experimenting with AWS and attempted to process 700K HTTP requests per minute on a replica of its live site, and Pinterest was growing massively on AWS . In th
5 0.95001674 1066 high scalability-2011-06-22-It's the Fraking IOPS - 1 SSD is 44,000 IOPS, Hard Drive is 180
Introduction: Planning your next buildout and thinking SSDs are still far in the future? Still too expensive, too low density. Hard disks are cheap, familiar, and store lots of stuff. In this short and entertaining video Wikia's Artur Bergman wants to change your mind about SSDs. SSDs are for today, get with the math already. Here's Artur's logic: Wikia is all SSD in production. The new Wikia file servers have a theoretical read rate of ~10GB/sec sequential, 6GB/sec random and 1.2 million IOPs. If you can't do math or love the past, you love spinning rust. If you are awesome you love SSDs. SSDs are cheaper than drives using the most relevant metric: $/GB/IOPS. 1 SSD is 44,000 IOPS and one hard drive is 180 IOPS. Need 1 SSD instead of 50 hard drives. With 8 million files there's a 9 minute fsck. Full backup in 12 minutes (X-25M based). 4 GB/sec random read average latency 1 msec. 2.2 GB/sec random write average latency 1 msec. 50TBs of SSDs in one machine for $80,000. With the densi
6 0.94271189 1299 high scalability-2012-08-06-Paper: High-Performance Concurrency Control Mechanisms for Main-Memory Databases
7 0.93667495 1635 high scalability-2014-04-21-This is why Microsoft won. And why they lost.
8 0.93543613 171 high scalability-2007-12-02-a8cjdbc - update verision 1.3
9 0.93542743 170 high scalability-2007-12-02-Database-Clustering: a8cjdbc - update: version 1.3
same-blog 10 0.92576444 584 high scalability-2009-04-27-Some Questions from a newbie
11 0.90884805 767 high scalability-2010-01-27-Hot Scalability Links for January 28 2010
12 0.90353829 1046 high scalability-2011-05-23-Evernote Architecture - 9 Million Users and 150 Million Requests a Day
13 0.89220423 792 high scalability-2010-03-10-How FarmVille Scales - The Follow-up
14 0.87809253 689 high scalability-2009-08-28-Strategy: Solve Only 80 Percent of the Problem
15 0.86571193 1631 high scalability-2014-04-14-How do you even do anything without using EBS?
16 0.8626911 1585 high scalability-2014-01-24-Stuff The Internet Says On Scalability For January 24th, 2014
17 0.82712674 142 high scalability-2007-11-05-Strategy: Diagonal Scaling - Don't Forget to Scale Out AND Up
18 0.80241007 1331 high scalability-2012-10-02-An Epic TripAdvisor Update: Why Not Run on the Cloud? The Grand Experiment.
19 0.80072927 1521 high scalability-2013-09-23-Salesforce Architecture - How they Handle 1.3 Billion Transactions a Day
20 0.79587579 257 high scalability-2008-02-22-Kevin's Great Adventures in SSDland