high_scalability high_scalability-2010 high_scalability-2010-792 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Several readers had follow-up questions in response to How FarmVille Scales to Harvest 75 Million Players a Month . Here are Luke's response to those questions (and a few of mine). How does social networking makes things easier or harder? The primary interesting aspect of social networking games is how you wind up with a graph of connected users who need to be access each other's data on a frequent basis. This makes the overall dataset difficult if not impossible to partition. What are examples of the Facebook calls you try to avoid and how they impact game play? We can make a call for facebook friend data to retrieve information about your friends playing the game. Normally, we show a friend ladder at the bottom of the game that shows friend information, including name and facebook photo. Can you say where your cache is, what form it takes, and how much cached there is? Do you have a peering relationship with Facebook, as one might expect at that bandwidth?
sentIndex sentText sentNum sentScore
1 The primary interesting aspect of social networking games is how you wind up with a graph of connected users who need to be access each other's data on a frequent basis. [sent-4, score-0.198]
2 What are examples of the Facebook calls you try to avoid and how they impact game play? [sent-6, score-0.372]
3 We can make a call for facebook friend data to retrieve information about your friends playing the game. [sent-7, score-0.687]
4 Normally, we show a friend ladder at the bottom of the game that shows friend information, including name and facebook photo. [sent-8, score-1.955]
5 What is the impact in game play when functionality is disabled in response to load? [sent-13, score-0.546]
6 What users will see is that some part of the application doesn't work as it normally does. [sent-14, score-0.198]
7 For example, on our neighbors page and in a friend ladder at the bottom of the flash application, you can see a list of your friend playing the game and their game stats. [sent-16, score-2.034]
8 It saves us some work on the backend and has a relatively small effect on user experience. [sent-18, score-0.238]
9 Do you try and stay out of there backend as much as possible? [sent-20, score-0.176]
10 It sounds like you can play most of the game in the client without talking to the backend for long periods of time. [sent-25, score-0.722]
11 I take it flows from the nature of the application where farms are relatively isolated from each other? [sent-26, score-0.235]
12 Is there any attempt to clump farms together as some other multiplayer games do? [sent-27, score-0.252]
13 Yes, one of the benefits of having an interactive client is that we have a bit of isolation between server latency and observed client latency. [sent-28, score-0.4]
14 We do verify each action performed in the game; however, we do it asynchronously and queue the transactions on the client. [sent-29, score-0.162]
15 The AMF transactions happen asynchronously from the client and if the server sees something it doesn't think the client should be sending, it returns to the client an "Out of Sync" message which tells the client it is in an invalid state and the client reloads itself. [sent-39, score-1.258]
16 An example inside of FarmVille is where we have a friend ladder inside the flash at the bottom of the page. [sent-47, score-1.173]
17 Normally, we query facebook for the name and profile picture and our own backend for the game stats and avatar data. [sent-48, score-0.922]
18 This is a high engagement piece of the application but lower in priority than users doing farming actions. [sent-49, score-0.198]
19 Thus, if our backend is having performance problems, we can turn that off and the friend ladder will only show facebook name and profile picture. [sent-50, score-1.586]
20 Likewise, if facebook is having performance problems, we can turn it off and the friend ladder will not show up. [sent-51, score-1.238]
wordName wordTfidf (topN-words)
[('ladder', 0.427), ('friend', 0.346), ('facebook', 0.254), ('game', 0.246), ('client', 0.2), ('backend', 0.176), ('bottom', 0.156), ('normally', 0.139), ('farmville', 0.136), ('turn', 0.128), ('peering', 0.121), ('flash', 0.118), ('farms', 0.114), ('asynchronously', 0.105), ('play', 0.1), ('name', 0.097), ('playing', 0.087), ('integrate', 0.086), ('degradable', 0.085), ('reloads', 0.085), ('tohow', 0.085), ('show', 0.083), ('xhr', 0.08), ('farming', 0.077), ('games', 0.075), ('profile', 0.075), ('builder', 0.074), ('avatar', 0.074), ('disabled', 0.071), ('response', 0.071), ('thus', 0.07), ('harvest', 0.07), ('examples', 0.068), ('use', 0.068), ('invalid', 0.068), ('luke', 0.068), ('likewise', 0.065), ('comet', 0.064), ('vertica', 0.064), ('inside', 0.063), ('wind', 0.063), ('multiplayer', 0.063), ('relatively', 0.062), ('engagement', 0.062), ('neighbors', 0.062), ('questions', 0.06), ('networking', 0.06), ('application', 0.059), ('impact', 0.058), ('verify', 0.057)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999994 792 high scalability-2010-03-10-How FarmVille Scales - The Follow-up
Introduction: Several readers had follow-up questions in response to How FarmVille Scales to Harvest 75 Million Players a Month . Here are Luke's response to those questions (and a few of mine). How does social networking makes things easier or harder? The primary interesting aspect of social networking games is how you wind up with a graph of connected users who need to be access each other's data on a frequent basis. This makes the overall dataset difficult if not impossible to partition. What are examples of the Facebook calls you try to avoid and how they impact game play? We can make a call for facebook friend data to retrieve information about your friends playing the game. Normally, we show a friend ladder at the bottom of the game that shows friend information, including name and facebook photo. Can you say where your cache is, what form it takes, and how much cached there is? Do you have a peering relationship with Facebook, as one might expect at that bandwidth?
2 0.26285619 774 high scalability-2010-02-08-How FarmVille Scales to Harvest 75 Million Players a Month
Introduction: Several readers had follow-up questions in response to this article. Luke's responses can be found in How FarmVille Scales - The Follow-up . If real farming was as comforting as it is in Zynga's mega-hit Farmville then my family would have probably never left those harsh North Dakota winters. None of the scary bedtime stories my Grandma used to tell about farming are true in FarmVille. Farmers make money, plants grow, and animals never visit the red barn . I guess it's just that keep-your-shoes-clean back-to-the-land charm that has helped make FarmVille the "largest game in the world" in such an astonishingly short time. How did FarmVille scale a web application to handle 75 million players a month? Fortunately FarmVille's Luke Rajlich has agreed to let us in on a few their challenges and secrets. Here's what Luke has to say... The format of the interview was that I sent Luke a few general questions and he replied with this response: FarmVille has a unique set of sc
3 0.21606661 904 high scalability-2010-09-21-Playfish's Social Gaming Architecture - 50 Million Monthly Users and Growing
Introduction: Ten million players a day and over fifty million players a month interact socially with friends using Playfish games on social platforms like The Facebook, MySpace, and the iPhone. Playfish was an early innovator in the fastest growing segment of the game industry: social gaming , which is the love child between casual gaming and social networking. Playfish was also an early adopter of the Amazon cloud, running their system entirely on 100s of cloud servers. Playfish finds itself at the nexus of some hot trends (which may by why EA bought them for $300 million and they think a $1 billion game is possible): building games on social networks, build applications in the cloud, mobile gaming, leveraging data driven design to continuously evolve and improve systems, agile development and deployment, and selling virtual good as a business model. How can a small company make all this happen? To explain the magic I interviewed Playfish's Jodi Moran, Senior Director of Engineering, an
4 0.17847601 337 high scalability-2008-05-31-memcached and Storage of Friend list
Introduction: My first post, please be gentle. I know it is long. You are all like doctors - the more info, the better the diagnosis. ----------- What is the best way to store a list of all of your friends in the memcached cache (a simple boolean saying “yes this user is your friend”, or “no”)? Think Robert Scoble (26,000+ “friends”) on Twitter.com. He views a list of ALL existing users, and in this list, his friends are highlighted. I came up with 4 possible methods: --store in memcache as an array, a list of all the "yes" friend ID's --store your friend ID's as individual elements. --store as a hash of arrays based on last 3 digits of friend's ID -- so have up to 1000 arrays for you. --comma-delimited string of ID's as one element I'm using the second one because I think it is faster to update. The single array or hash of arrays feels like too much overhead calculating and updating – and even just loading – to check for existence of a friend. The key is FRIEND[small ID#]_
5 0.17103866 1405 high scalability-2013-02-13-7 Sensible and 1 Really Surprising Way EVE Online Scales to Play Huge Games
Introduction: "Everything in war is simple, but the simplest thing is difficult." -- Carl von Clausewitz Games are proving grounds for software architecture. They combine scale, high performance, challenging problems, a rabid user base, cost sensitivity, and the need for profit. And when games have in-game currency, like EVE Online has, there's money at play, so you can't just get away with a c'est la vie attitude. Engineering must be applied. In Planning for war: how the EVE Online servers deal with a 3,000 person battle , we learn some techniques EVE Online uses to handle large games: 7 Sensible... Do nothing. Most games are manageable or have spikes that quickly dissipate. Run it Hot . There's nothing to throttle as servers run at 100%. Why waste money? Use all your CPU. Shard it . Games are sharded by solar system and multiple solar systems run on a node. Move it . Games are moved when a machine becomes overloaded. Live Node Remap, where a live game is moved to
6 0.15551615 422 high scalability-2008-10-17-Scaling Spam Eradication Using Purposeful Games: Die Spammer Die!
7 0.15263289 1044 high scalability-2011-05-19-Zynga's Z Cloud - Scale Fast or Fail Fast by Merging Private and Public Clouds
8 0.14975259 1123 high scalability-2011-09-23-The Real News is Not that Facebook Serves Up 1 Trillion Pages a Month…
9 0.1480604 721 high scalability-2009-10-13-Why are Facebook, Digg, and Twitter so hard to scale?
10 0.14660616 496 high scalability-2009-01-17-Scaling in Games & Virtual Worlds
11 0.14475188 720 high scalability-2009-10-12-High Performance at Massive Scale – Lessons learned at Facebook
12 0.13927373 205 high scalability-2008-01-10-Letting Clients Know What's Changed: Push Me or Pull Me?
13 0.13892394 1342 high scalability-2012-10-17-World of Warcraft's Lead designer Rob Pardo on the Role of the Cloud in Games
14 0.13824755 845 high scalability-2010-06-22-Exploring the software behind Facebook, the world’s largest site
15 0.12723596 624 high scalability-2009-06-10-Hive - A Petabyte Scale Data Warehouse using Hadoop
16 0.12293516 840 high scalability-2010-06-10-The Four Meta Secrets of Scaling at Facebook
17 0.11428183 313 high scalability-2008-05-02-Friends for Sale Architecture - A 300 Million Page View-Month Facebook RoR App
18 0.11051365 406 high scalability-2008-10-08-Strategy: Flickr - Do the Essential Work Up-front and Queue the Rest
19 0.11015654 129 high scalability-2007-10-23-Hire Facebook, Ning, and Salesforce to Scale for You
topicId topicWeight
[(0, 0.175), (1, 0.081), (2, 0.063), (3, -0.055), (4, 0.081), (5, -0.048), (6, -0.016), (7, 0.068), (8, -0.022), (9, -0.039), (10, 0.136), (11, 0.091), (12, 0.041), (13, 0.058), (14, -0.064), (15, 0.034), (16, 0.077), (17, 0.055), (18, 0.048), (19, 0.014), (20, 0.065), (21, 0.046), (22, 0.078), (23, 0.01), (24, 0.049), (25, 0.026), (26, 0.071), (27, -0.058), (28, 0.029), (29, -0.083), (30, -0.036), (31, -0.003), (32, -0.032), (33, -0.04), (34, 0.047), (35, 0.018), (36, 0.136), (37, -0.106), (38, 0.015), (39, -0.086), (40, 0.019), (41, -0.028), (42, 0.028), (43, 0.058), (44, -0.034), (45, -0.001), (46, -0.023), (47, -0.027), (48, 0.012), (49, -0.037)]
simIndex simValue blogId blogTitle
same-blog 1 0.97205859 792 high scalability-2010-03-10-How FarmVille Scales - The Follow-up
Introduction: Several readers had follow-up questions in response to How FarmVille Scales to Harvest 75 Million Players a Month . Here are Luke's response to those questions (and a few of mine). How does social networking makes things easier or harder? The primary interesting aspect of social networking games is how you wind up with a graph of connected users who need to be access each other's data on a frequent basis. This makes the overall dataset difficult if not impossible to partition. What are examples of the Facebook calls you try to avoid and how they impact game play? We can make a call for facebook friend data to retrieve information about your friends playing the game. Normally, we show a friend ladder at the bottom of the game that shows friend information, including name and facebook photo. Can you say where your cache is, what form it takes, and how much cached there is? Do you have a peering relationship with Facebook, as one might expect at that bandwidth?
2 0.79201877 774 high scalability-2010-02-08-How FarmVille Scales to Harvest 75 Million Players a Month
Introduction: Several readers had follow-up questions in response to this article. Luke's responses can be found in How FarmVille Scales - The Follow-up . If real farming was as comforting as it is in Zynga's mega-hit Farmville then my family would have probably never left those harsh North Dakota winters. None of the scary bedtime stories my Grandma used to tell about farming are true in FarmVille. Farmers make money, plants grow, and animals never visit the red barn . I guess it's just that keep-your-shoes-clean back-to-the-land charm that has helped make FarmVille the "largest game in the world" in such an astonishingly short time. How did FarmVille scale a web application to handle 75 million players a month? Fortunately FarmVille's Luke Rajlich has agreed to let us in on a few their challenges and secrets. Here's what Luke has to say... The format of the interview was that I sent Luke a few general questions and he replied with this response: FarmVille has a unique set of sc
Introduction: Facebook has been teasing us. While many of their recent acquisitions have been surprising, shocking is the only word adequately describing Facebook's 5 day whirlwind acquisition of Oculus , immersive virtual reality visionaries, for a now paltry sounding $2 billion. The backlash is a pandemic, jumping across social networks with the speed only a meme powered by the directly unaffected can generate. For more than 30 years VR has been the dream burning in the heart of every science fiction fan. Now that this future might finally be here, Facebook’s ownage makes it seem like a wonderful and hopeful timeline has been choked off, killing the Metaverse before it even had a chance to begin. For the many who voted for an open future with their Kickstarter dollars , there’s a deep and personal sense of betrayal, despite Facebook’s promise to leave Oculus alone. The intensity of the reaction is because Oculus matters to people. It's new, it's different, it create
4 0.72073579 1168 high scalability-2012-01-04-How Facebook Handled the New Year's Eve Onslaught
Introduction: How does Facebook handle the massive New Year's Eve traffic spike? Thanks to Mike Swift, in Facebook gets ready for New Year's Eve , we get a little insight as to their method for the madness, nothing really detailed, but still interesting. Problem Setup Facebook expects tha one billion+ photos will be shared on New Year's eve. Facebook's 800 million users are scattered around the world. Three quarters live outside the US. Each user is linked to an average of 130 friends. Photos and posts must appear in less than a second. Opening a homepage requires executing requests on a 100 different servers, and those requests have to be ranked, sorted, and privacy-checked, and then rendered. Different events put different stresses on different parts of Facebook. Photo and Video Uploads - Holidays require hundreds of terabytes of capacity News Feed - News events like big sports events and the death of Steve Jobs drive user status updates Coping Strategies Try
5 0.70357382 904 high scalability-2010-09-21-Playfish's Social Gaming Architecture - 50 Million Monthly Users and Growing
Introduction: Ten million players a day and over fifty million players a month interact socially with friends using Playfish games on social platforms like The Facebook, MySpace, and the iPhone. Playfish was an early innovator in the fastest growing segment of the game industry: social gaming , which is the love child between casual gaming and social networking. Playfish was also an early adopter of the Amazon cloud, running their system entirely on 100s of cloud servers. Playfish finds itself at the nexus of some hot trends (which may by why EA bought them for $300 million and they think a $1 billion game is possible): building games on social networks, build applications in the cloud, mobile gaming, leveraging data driven design to continuously evolve and improve systems, agile development and deployment, and selling virtual good as a business model. How can a small company make all this happen? To explain the magic I interviewed Playfish's Jodi Moran, Senior Director of Engineering, an
6 0.70137107 1405 high scalability-2013-02-13-7 Sensible and 1 Really Surprising Way EVE Online Scales to Play Huge Games
7 0.66886926 318 high scalability-2008-05-14-New Facebook Chat Feature Scales to 70 Million Users Using Erlang
8 0.66664028 496 high scalability-2009-01-17-Scaling in Games & Virtual Worlds
9 0.64763075 1602 high scalability-2014-02-26-The WhatsApp Architecture Facebook Bought For $19 Billion
10 0.64249104 870 high scalability-2010-08-02-7 Scaling Strategies Facebook Used to Grow to 500 Million Users
11 0.63775003 424 high scalability-2008-10-22-EVE Online Architecture
12 0.63460058 1323 high scalability-2012-09-15-4 Reasons Facebook Dumped HTML5 and Went Native
13 0.63014847 966 high scalability-2010-12-31-Facebook in 20 Minutes: 2.7M Photos, 10.2M Comments, 4.6M Messages
14 0.62576646 1123 high scalability-2011-09-23-The Real News is Not that Facebook Serves Up 1 Trillion Pages a Month…
15 0.62350857 422 high scalability-2008-10-17-Scaling Spam Eradication Using Purposeful Games: Die Spammer Die!
16 0.61742431 464 high scalability-2008-12-13-Strategy: Facebook Tweaks to Handle 6 Time as Many Memcached Requests
17 0.60972911 1342 high scalability-2012-10-17-World of Warcraft's Lead designer Rob Pardo on the Role of the Cloud in Games
18 0.60269928 845 high scalability-2010-06-22-Exploring the software behind Facebook, the world’s largest site
19 0.59890157 720 high scalability-2009-10-12-High Performance at Massive Scale – Lessons learned at Facebook
20 0.59651864 840 high scalability-2010-06-10-The Four Meta Secrets of Scaling at Facebook
topicId topicWeight
[(1, 0.128), (2, 0.175), (10, 0.372), (22, 0.046), (30, 0.05), (61, 0.06), (77, 0.015), (79, 0.058)]
simIndex simValue blogId blogTitle
1 0.97280431 430 high scalability-2008-10-26-Should you use a SAN to scale your architecture?
Introduction: This is a question everyone must struggle with when building out their datacenter. Storage choices are always the ones I have the least confidence in. David Marks in his blog You Can Change It Later! asks the question Should I get a SAN to scale my site architecture? and answers no. A better solution is to use commodity hardware, directly attach storage on servers, and partition across servers to scale and for greater availability. David's reasoning is interesting: A SAN creates a SPOF (single point of failure) that is dependent on a vendor to fly and fix when there's a problem. This can lead to long down times during this outage you have no access to your data at all. Using easily available commodity hardware minimizes risks to your company, it's not just about saving money. Zooming over to Fry's to buy emergency equipment provides the kind of agility startups need in order to respond quickly to ever changing situations. It's hard to beat the power and flexibility (backup
Introduction: This is a guest post by Ali Khajeh-Hosseini , Technical Lead at PlanForCloud . The original article was published on their site . With 29 cloud price reductions I thought it would be interesting to see how the bottom line would change compared to an article we published last year . The result is surprisingly little for TripAdvisor because prices for On Demand instances have not dropped as fast as for other other instances types. Over the last year and a half, we counted 29 price reductions in cloud services provided by AWS, Google Compute Engine, Windows Azure, and Rackspace Cloud. Price reductions have a direct effect on cloud users, but given the usual tiny reductions, how significant is that effect on the bottom line? Last year I wrote about cloud cost forecasts for TripAdvisor and Pinterest . TripAdvisor was experimenting with AWS and attempted to process 700K HTTP requests per minute on a replica of its live site, and Pinterest was growing massively on AWS . In th
3 0.971035 1066 high scalability-2011-06-22-It's the Fraking IOPS - 1 SSD is 44,000 IOPS, Hard Drive is 180
Introduction: Planning your next buildout and thinking SSDs are still far in the future? Still too expensive, too low density. Hard disks are cheap, familiar, and store lots of stuff. In this short and entertaining video Wikia's Artur Bergman wants to change your mind about SSDs. SSDs are for today, get with the math already. Here's Artur's logic: Wikia is all SSD in production. The new Wikia file servers have a theoretical read rate of ~10GB/sec sequential, 6GB/sec random and 1.2 million IOPs. If you can't do math or love the past, you love spinning rust. If you are awesome you love SSDs. SSDs are cheaper than drives using the most relevant metric: $/GB/IOPS. 1 SSD is 44,000 IOPS and one hard drive is 180 IOPS. Need 1 SSD instead of 50 hard drives. With 8 million files there's a 9 minute fsck. Full backup in 12 minutes (X-25M based). 4 GB/sec random read average latency 1 msec. 2.2 GB/sec random write average latency 1 msec. 50TBs of SSDs in one machine for $80,000. With the densi
4 0.96641302 1299 high scalability-2012-08-06-Paper: High-Performance Concurrency Control Mechanisms for Main-Memory Databases
Introduction: If you stayed up all night watching the life reaffirming Curiosity landing on Mars , then this paper, High-Performance Concurrency Control Mechanisms for Main-Memory Databases , has nothing to do with that at all, but it is an excellent look at how to use optimistic MVCC schemes to reduce lock overhead on in-memory datastructures: A database system optimized for in-memory storage can support much higher transaction rates than current systems. However, standard concurrency control methods used today do not scale to the high transaction rates achievable by such systems. In this paper we introduce two efficient concurrency control methods specifically designed for main-memory databases. Both use multiversioning to isolate read-only transactions from updates but differ in how atomicity is ensured: one is optimistic and one is pessimistic. To avoid expensive context switching, transactions never block during normal processing but they may have to wait before commit to ensure corr
5 0.95931435 1635 high scalability-2014-04-21-This is why Microsoft won. And why they lost.
Introduction: My favorite kind of histories are those told from an insider's perspective. The story of Richard the Lionheart is full of great battles and dynastic intrigue. The story of one of his soldiers, not so much. Yet the soldiers' story, as someone who has experienced the real consequences of decisions made and actions taken, is more revealing. We get such a history in Chat Wars , a wonderful article written by David Auerbach, who in 1998 worked at Microsoft on MSN Messenger Service, Microsoft’s instant messaging app (for a related story see The Rise and Fall of AIM, the Breakthrough AOL Never Wanted ). It's as if Herodotus visited Microsoft and wrote down his experiences. It has that same sort of conversational tone, insightful on-the-ground observations, and facts no outsider might ever believe. Much of the article is a play-by-play account of the cat and mouse game David plays changing Messenger to track AOL's Instant Messenger protocol changes. AOL repeatedly tried to make it so M
6 0.9527365 874 high scalability-2010-08-07-ArchCamp: Scalable Databases (NoSQL)
7 0.93851429 767 high scalability-2010-01-27-Hot Scalability Links for January 28 2010
8 0.93333304 171 high scalability-2007-12-02-a8cjdbc - update verision 1.3
9 0.93331891 170 high scalability-2007-12-02-Database-Clustering: a8cjdbc - update: version 1.3
same-blog 10 0.9290325 792 high scalability-2010-03-10-How FarmVille Scales - The Follow-up
11 0.92763996 1046 high scalability-2011-05-23-Evernote Architecture - 9 Million Users and 150 Million Requests a Day
12 0.92367655 178 high scalability-2007-12-10-1 Master, N Slaves
13 0.92074203 584 high scalability-2009-04-27-Some Questions from a newbie
14 0.90862149 1631 high scalability-2014-04-14-How do you even do anything without using EBS?
15 0.89742792 1585 high scalability-2014-01-24-Stuff The Internet Says On Scalability For January 24th, 2014
16 0.88724679 689 high scalability-2009-08-28-Strategy: Solve Only 80 Percent of the Problem
17 0.84623152 1331 high scalability-2012-10-02-An Epic TripAdvisor Update: Why Not Run on the Cloud? The Grand Experiment.
18 0.84594518 142 high scalability-2007-11-05-Strategy: Diagonal Scaling - Don't Forget to Scale Out AND Up
19 0.8393054 1353 high scalability-2012-11-01-Cost Analysis: TripAdvisor and Pinterest costs on the AWS cloud
20 0.82997417 257 high scalability-2008-02-22-Kevin's Great Adventures in SSDland