high_scalability high_scalability-2009 high_scalability-2009-496 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: "Online games and virtual worlds have familiar scaling requirements, but don’t be fooled: everything you know is wrong." Jim Waldo, Sun Microsystems Laboratories * The computational environment for online games or virtual worlds is close to the exact inverse of that found in most markets serviced by the high-tech industry. * The need for a heavyweight client is, in part, an outcome of the evolution of these games. * Latency is the enemy of fun—and therefore the enemy of online games and virtual worlds. * The game server is used both to discourage cheating (by making it much more difficult) and to detect cheating (by seeing patterns of divergence between the game state reported by the client and the game state held by the server). Peer-to-peer technologies might seem a natural fit for the first role of the game server, but this second role means that few if any games or worlds trust their peers enough to avoid the server component. * Using multiple servers is a basic mecha
sentIndex sentText sentNum sentScore
1 "Online games and virtual worlds have familiar scaling requirements, but don’t be fooled: everything you know is wrong. [sent-1, score-0.804]
2 " Jim Waldo, Sun Microsystems Laboratories * The computational environment for online games or virtual worlds is close to the exact inverse of that found in most markets serviced by the high-tech industry. [sent-2, score-1.094]
3 * The need for a heavyweight client is, in part, an outcome of the evolution of these games. [sent-3, score-0.277]
4 * Latency is the enemy of fun—and therefore the enemy of online games and virtual worlds. [sent-4, score-0.912]
5 * The game server is used both to discourage cheating (by making it much more difficult) and to detect cheating (by seeing patterns of divergence between the game state reported by the client and the game state held by the server). [sent-5, score-1.862]
6 Peer-to-peer technologies might seem a natural fit for the first role of the game server, but this second role means that few if any games or worlds trust their peers enough to avoid the server component. [sent-6, score-1.255]
7 * Using multiple servers is a basic mechanism for scaling the server component of a game to the levels that are being seen in the online world today. [sent-7, score-0.656]
8 * Having multiple servers means that part of building the game is deciding how to partition the load over these servers. [sent-8, score-0.396]
9 The first technique is to exploit the geography of the game or world. [sent-9, score-0.722]
10 * The problem is that the culture that has grown up around games and virtual worlds is not one that understands or is overly familiar with the programming techniques that are required to exploit the parallelism inherent in these systems. [sent-12, score-1.388]
11 *The model is a simple event-based one in which input from the client is received by the server, which then sets off a task in response to that event. [sent-16, score-0.169]
12 * This mechanism for concurrency control does require that all tasks access all of their data through the Darkstar data service. [sent-17, score-0.102]
13 we believe that we can keep the penalty for accessing through a data service small by caching data in intelligent ways. [sent-19, score-0.299]
14 We also believe that by using the inherent parallelism in these games, we can increase the overall performance of the game as the number of players increases, even if there is a small penalty for individual data access. [sent-20, score-0.878]
15 * We found that additional machines lowered the capacity of the overall system. [sent-21, score-0.179]
16 We are working on removing the choke points so that adding equipment actually adds capacity. [sent-22, score-0.086]
wordName wordTfidf (topN-words)
[('game', 0.325), ('games', 0.298), ('worlds', 0.28), ('darkstar', 0.25), ('cheating', 0.203), ('exploit', 0.195), ('enemy', 0.176), ('penalty', 0.152), ('online', 0.139), ('virtual', 0.123), ('inherent', 0.119), ('technique', 0.114), ('fooled', 0.113), ('laboratories', 0.113), ('waldo', 0.113), ('parallelism', 0.11), ('inverse', 0.106), ('familiar', 0.103), ('mechanism', 0.102), ('client', 0.099), ('role', 0.095), ('divergence', 0.095), ('overly', 0.092), ('lowered', 0.092), ('server', 0.09), ('heavyweight', 0.09), ('geography', 0.088), ('illusion', 0.088), ('outcome', 0.088), ('overall', 0.087), ('choke', 0.086), ('microsystems', 0.086), ('believe', 0.085), ('presenting', 0.084), ('serviced', 0.084), ('attempting', 0.079), ('multithreaded', 0.078), ('peers', 0.072), ('berkeley', 0.071), ('deciding', 0.071), ('received', 0.07), ('jim', 0.07), ('reported', 0.069), ('produced', 0.068), ('understands', 0.068), ('multicore', 0.067), ('markets', 0.064), ('chips', 0.064), ('state', 0.064), ('accessing', 0.062)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999994 496 high scalability-2009-01-17-Scaling in Games & Virtual Worlds
Introduction: "Online games and virtual worlds have familiar scaling requirements, but don’t be fooled: everything you know is wrong." Jim Waldo, Sun Microsystems Laboratories * The computational environment for online games or virtual worlds is close to the exact inverse of that found in most markets serviced by the high-tech industry. * The need for a heavyweight client is, in part, an outcome of the evolution of these games. * Latency is the enemy of fun—and therefore the enemy of online games and virtual worlds. * The game server is used both to discourage cheating (by making it much more difficult) and to detect cheating (by seeing patterns of divergence between the game state reported by the client and the game state held by the server). Peer-to-peer technologies might seem a natural fit for the first role of the game server, but this second role means that few if any games or worlds trust their peers enough to avoid the server component. * Using multiple servers is a basic mecha
2 0.30909029 904 high scalability-2010-09-21-Playfish's Social Gaming Architecture - 50 Million Monthly Users and Growing
Introduction: Ten million players a day and over fifty million players a month interact socially with friends using Playfish games on social platforms like The Facebook, MySpace, and the iPhone. Playfish was an early innovator in the fastest growing segment of the game industry: social gaming , which is the love child between casual gaming and social networking. Playfish was also an early adopter of the Amazon cloud, running their system entirely on 100s of cloud servers. Playfish finds itself at the nexus of some hot trends (which may by why EA bought them for $300 million and they think a $1 billion game is possible): building games on social networks, build applications in the cloud, mobile gaming, leveraging data driven design to continuously evolve and improve systems, agile development and deployment, and selling virtual good as a business model. How can a small company make all this happen? To explain the magic I interviewed Playfish's Jodi Moran, Senior Director of Engineering, an
3 0.28130084 1342 high scalability-2012-10-17-World of Warcraft's Lead designer Rob Pardo on the Role of the Cloud in Games
Introduction: In a really far ranging and insightful interview by Steve Peterson: Game Industry Legends: Rob Pardo , where the future of gaming is discussed, there was a section on how the cloud might be used in games. I know there are a lot of game developers in my audience, so I thought it might be useful: Q. If the game is free-to-play but I have to download 10 gigabytes to try it out, that can keep me from trying it. That's part of what cloud gaming is trying to overcome; do you think cloud gaming is going to make some inroads because of those technical issues? Rob Pardo : I certainly think there's a lot of potential in cloud gaming, it's just picking the right games. There's a lot of hurdles for cloud gaming to overcome, one of which is having to have all these servers to be able to host all these games, since it's not like you need any less computing power to play these games. First of all you need someone with the infrastructure that can have all those games up there. Another issue t
4 0.26016653 1405 high scalability-2013-02-13-7 Sensible and 1 Really Surprising Way EVE Online Scales to Play Huge Games
Introduction: "Everything in war is simple, but the simplest thing is difficult." -- Carl von Clausewitz Games are proving grounds for software architecture. They combine scale, high performance, challenging problems, a rabid user base, cost sensitivity, and the need for profit. And when games have in-game currency, like EVE Online has, there's money at play, so you can't just get away with a c'est la vie attitude. Engineering must be applied. In Planning for war: how the EVE Online servers deal with a 3,000 person battle , we learn some techniques EVE Online uses to handle large games: 7 Sensible... Do nothing. Most games are manageable or have spikes that quickly dissipate. Run it Hot . There's nothing to throttle as servers run at 100%. Why waste money? Use all your CPU. Shard it . Games are sharded by solar system and multiple solar systems run on a node. Move it . Games are moved when a machine becomes overloaded. Live Node Remap, where a live game is moved to
5 0.22931735 422 high scalability-2008-10-17-Scaling Spam Eradication Using Purposeful Games: Die Spammer Die!
Introduction: Update: As expected I'm undergoing a massive spam attack for speaking truth to dark powers. This is the time to be strong. Together we can make a change. What change you may ask? I can't say, just change and lots more change. Let's link arms together and bravely stand against the forces of chaos for a better yesterday and a better tomorrow. CAPTCHA doesn't work. Even Google can't make CAPTCHA work ( Spammers Choose GMail ). And even if CAPTCHA worked it wouldn't really work because CAPTCHA solving markets ( Inside India’s CAPTCHA solving economy ) have evolved where for a mere $2 you can buy 1000 human broken CAPTCHA's. And we know once the free market tackles a problem that's it. Game over :-) Making ever more clever CAPTCHA programs won't outwit and outlast the CAPTCHA solving markets. Until Skynet evolves the only way to defeat humans is with humans. Using Games to Get Humans to Do Work (like CAPTCHA) for Free How do we harness the power of humans to do battle with the C
6 0.17296112 1044 high scalability-2011-05-19-Zynga's Z Cloud - Scale Fast or Fail Fast by Merging Private and Public Clouds
7 0.17294878 755 high scalability-2009-12-28-Zynga Needs a Server-side Systems Engineer
9 0.14660616 792 high scalability-2010-03-10-How FarmVille Scales - The Follow-up
10 0.14545313 969 high scalability-2011-01-04-Sponsored Post: Newrelic, Cloudkick, Strata, EA, Joyent, CloudSigma, ManageEngine, Site24x7
11 0.13807538 424 high scalability-2008-10-22-EVE Online Architecture
topicId topicWeight
[(0, 0.154), (1, 0.039), (2, 0.113), (3, -0.012), (4, 0.039), (5, -0.028), (6, 0.034), (7, 0.026), (8, -0.082), (9, -0.033), (10, 0.138), (11, 0.004), (12, 0.027), (13, 0.104), (14, 0.035), (15, 0.019), (16, 0.063), (17, 0.075), (18, 0.021), (19, -0.005), (20, 0.048), (21, -0.077), (22, -0.045), (23, 0.001), (24, -0.02), (25, 0.06), (26, 0.035), (27, -0.054), (28, -0.023), (29, -0.043), (30, 0.118), (31, 0.106), (32, -0.088), (33, -0.07), (34, 0.039), (35, -0.045), (36, 0.15), (37, -0.079), (38, 0.066), (39, -0.137), (40, -0.003), (41, -0.037), (42, -0.028), (43, 0.048), (44, 0.05), (45, 0.021), (46, 0.042), (47, -0.063), (48, 0.041), (49, 0.011)]
simIndex simValue blogId blogTitle
same-blog 1 0.95743006 496 high scalability-2009-01-17-Scaling in Games & Virtual Worlds
Introduction: "Online games and virtual worlds have familiar scaling requirements, but don’t be fooled: everything you know is wrong." Jim Waldo, Sun Microsystems Laboratories * The computational environment for online games or virtual worlds is close to the exact inverse of that found in most markets serviced by the high-tech industry. * The need for a heavyweight client is, in part, an outcome of the evolution of these games. * Latency is the enemy of fun—and therefore the enemy of online games and virtual worlds. * The game server is used both to discourage cheating (by making it much more difficult) and to detect cheating (by seeing patterns of divergence between the game state reported by the client and the game state held by the server). Peer-to-peer technologies might seem a natural fit for the first role of the game server, but this second role means that few if any games or worlds trust their peers enough to avoid the server component. * Using multiple servers is a basic mecha
2 0.90322357 1405 high scalability-2013-02-13-7 Sensible and 1 Really Surprising Way EVE Online Scales to Play Huge Games
Introduction: "Everything in war is simple, but the simplest thing is difficult." -- Carl von Clausewitz Games are proving grounds for software architecture. They combine scale, high performance, challenging problems, a rabid user base, cost sensitivity, and the need for profit. And when games have in-game currency, like EVE Online has, there's money at play, so you can't just get away with a c'est la vie attitude. Engineering must be applied. In Planning for war: how the EVE Online servers deal with a 3,000 person battle , we learn some techniques EVE Online uses to handle large games: 7 Sensible... Do nothing. Most games are manageable or have spikes that quickly dissipate. Run it Hot . There's nothing to throttle as servers run at 100%. Why waste money? Use all your CPU. Shard it . Games are sharded by solar system and multiple solar systems run on a node. Move it . Games are moved when a machine becomes overloaded. Live Node Remap, where a live game is moved to
3 0.85615414 424 high scalability-2008-10-22-EVE Online Architecture
Introduction: Sorry, the content for this post apparently did not make the transition from the old HighScalability website, it's all messed up, but there's still a some useful content... EVE Online is "The World's Largest Game Universe", a massively multiplayer online game ( MMO ) made by CCP. EVE Online's Architecture is unusual for a MMOG because it doesn't divide the player load among different servers or shards. Instead, the same cluster handles the entire EVE universe. It is an interesting to compare this with the Architecture of the Second Life Grid . How do they manage to scale? Information Sources EVE Insider Dev Blog EVE Online FAQ Massively - Eve Evolved: Eve Online's Server Model and its discussion on Slashdot EVE Online Forums Massively Multiplayer Game Development 2 Platform Stackless Python used for both server and client game logic. It allows programmers to reap the benefits of thread-based programming without the performance and complexity p
4 0.81607598 422 high scalability-2008-10-17-Scaling Spam Eradication Using Purposeful Games: Die Spammer Die!
Introduction: Update: As expected I'm undergoing a massive spam attack for speaking truth to dark powers. This is the time to be strong. Together we can make a change. What change you may ask? I can't say, just change and lots more change. Let's link arms together and bravely stand against the forces of chaos for a better yesterday and a better tomorrow. CAPTCHA doesn't work. Even Google can't make CAPTCHA work ( Spammers Choose GMail ). And even if CAPTCHA worked it wouldn't really work because CAPTCHA solving markets ( Inside India’s CAPTCHA solving economy ) have evolved where for a mere $2 you can buy 1000 human broken CAPTCHA's. And we know once the free market tackles a problem that's it. Game over :-) Making ever more clever CAPTCHA programs won't outwit and outlast the CAPTCHA solving markets. Until Skynet evolves the only way to defeat humans is with humans. Using Games to Get Humans to Do Work (like CAPTCHA) for Free How do we harness the power of humans to do battle with the C
5 0.80276161 1342 high scalability-2012-10-17-World of Warcraft's Lead designer Rob Pardo on the Role of the Cloud in Games
Introduction: In a really far ranging and insightful interview by Steve Peterson: Game Industry Legends: Rob Pardo , where the future of gaming is discussed, there was a section on how the cloud might be used in games. I know there are a lot of game developers in my audience, so I thought it might be useful: Q. If the game is free-to-play but I have to download 10 gigabytes to try it out, that can keep me from trying it. That's part of what cloud gaming is trying to overcome; do you think cloud gaming is going to make some inroads because of those technical issues? Rob Pardo : I certainly think there's a lot of potential in cloud gaming, it's just picking the right games. There's a lot of hurdles for cloud gaming to overcome, one of which is having to have all these servers to be able to host all these games, since it's not like you need any less computing power to play these games. First of all you need someone with the infrastructure that can have all those games up there. Another issue t
6 0.75633848 904 high scalability-2010-09-21-Playfish's Social Gaming Architecture - 50 Million Monthly Users and Growing
7 0.65856338 1044 high scalability-2011-05-19-Zynga's Z Cloud - Scale Fast or Fail Fast by Merging Private and Public Clouds
8 0.63210362 755 high scalability-2009-12-28-Zynga Needs a Server-side Systems Engineer
9 0.57511431 792 high scalability-2010-03-10-How FarmVille Scales - The Follow-up
10 0.53537202 1524 high scalability-2013-09-30-Expandability: Steve Wozniak's Biggest Success and Nolan Bushnell's Biggest Regret
11 0.51593047 774 high scalability-2010-02-08-How FarmVille Scales to Harvest 75 Million Players a Month
12 0.50361747 1412 high scalability-2013-02-25-SongPop Scales to 1 Million Active Users on GAE, Showing PaaS is not Passé
13 0.46361554 1075 high scalability-2011-07-07-Myth: Google Uses Server Farms So You Should Too - Resurrection of the Big-Ass Machines
14 0.4621692 1377 high scalability-2012-12-26-Ask HS: What will programming and architecture look like in 2020?
15 0.44956437 1592 high scalability-2014-02-07-Stuff The Internet Says On Scalability For February 7th, 2014
16 0.44541496 473 high scalability-2008-12-20-Second Life Architecture - The Grid
17 0.43436405 1397 high scalability-2013-02-01-Stuff The Internet Says On Scalability For February 1, 2013
18 0.43255922 953 high scalability-2010-12-03-GPU vs CPU Smackdown : The Rise of Throughput-Oriented Architectures
19 0.43192875 962 high scalability-2010-12-21-Sponsored Post: Electronic Arts, Joyent, Membase, CloudSigma, ManageEngine, Site24x7
20 0.42892173 1610 high scalability-2014-03-11-Douglas Adams - 3 Rules that Describe Our Reactions to Technologies
topicId topicWeight
[(1, 0.195), (2, 0.188), (10, 0.059), (30, 0.015), (47, 0.018), (48, 0.153), (61, 0.096), (73, 0.015), (79, 0.099), (85, 0.012), (94, 0.048)]
simIndex simValue blogId blogTitle
1 0.93642819 610 high scalability-2009-05-29-Is Eucalyptus ready to be your private cloud?
Introduction: Update: : Eucalyptus Goes Commercial with $5.5M Funding Round . This removes my objection that it's an academic project only. Go team go! Rich Wolski , professor of Computer Science at the University of California, Santa Barbara, gave a spirited talk on Eucalyptus to a large group of very interested cloudsters at the Eucalyptus Cloud Meetup . If Rich could teach computer science at every school the state of the computer science industry would be stratospheric. Rich is dynamic, smart, passionate, and visionary. It's that vision that prompted him to create Eucalyptus in the first place. Rich and his group are experts in grid and distributed computing, having a long and glorious history in that space. When he saw cloud computing on the rise he decided the best way to explore it was to implement what everyone accepted as a real cloud, Amazon's API. In a remarkably short time they implement Eucalyptus and have been improving it and tracking Amazon's changes ever since. The question
same-blog 2 0.93422866 496 high scalability-2009-01-17-Scaling in Games & Virtual Worlds
Introduction: "Online games and virtual worlds have familiar scaling requirements, but don’t be fooled: everything you know is wrong." Jim Waldo, Sun Microsystems Laboratories * The computational environment for online games or virtual worlds is close to the exact inverse of that found in most markets serviced by the high-tech industry. * The need for a heavyweight client is, in part, an outcome of the evolution of these games. * Latency is the enemy of fun—and therefore the enemy of online games and virtual worlds. * The game server is used both to discourage cheating (by making it much more difficult) and to detect cheating (by seeing patterns of divergence between the game state reported by the client and the game state held by the server). Peer-to-peer technologies might seem a natural fit for the first role of the game server, but this second role means that few if any games or worlds trust their peers enough to avoid the server component. * Using multiple servers is a basic mecha
3 0.93148273 611 high scalability-2009-05-31-Need help on Site loading & database optimization - URGENT
Introduction: Hi Friends, I need some help in making site access fast. On an average my site has the traffic 2500 hits per day and on 16th May it had 60,000 hits. On this day site was loading very slow even it was getting time out. I also check out the processes running by using "top" command it was indicating mysql was taking too much load. There are around 166 tables (Including PHPBB forum) in my database. All contents on site are displayed by fetching it from database. I have also added indexing to respective tables where it is required. Plain PHP/HTML coding is used. Technology: PHP -- 5.2 MYSQL -- 5.0 Apache -- 2.0 Linux Following is all the server details of my site: CPU : Single Socket Dual Core AMD Opteron 1212HE Memory: 2GB DDR RAM Hard Drive: 250GB SATA Ethernet: 100Mb Primary Ethernet Card (/var/log) # uname -a Linux 2.6.9-67.0.15.ELsmp #1 SMP Tue Apr 22 13:50:33 EDT 2008 i686 athlon i386 GNU/Linux kernel version: 2.6.9-67.0.15.ELsmp (/var/log) # free -m total used
4 0.92857212 880 high scalability-2010-08-13-Hot Scalability Links for Aug 13, 2010
Introduction: Ezra Zygmuntowicz in a heart warming account of his 4 Years at Engine Yard , has concluded in his experience that: the true future of cloud computing for developers is to not think about servers at all. It is now time to focus on the Application and new levels of abstraction that allow folks to use the computing resources in easier and easier ways. Tweets of Gold: bryanlatten : Nothing like a million caching layers to screw up an already complicated deployment. Thankfully, there is beer. jkalucki : Twitter isn't down, you are just using the wrong access methods... andyedinborough : I don't mean to hate, but why would I give up performance and scalability for a dynamic language? Honestly, I don't get it. AsitSinha : It's amazing.... to see the absence of an understanding of how capability plays a role in scalability. scottgal : Devs are HORRIBLE DBAs. Used to do Scalability labs for MS UK and bad schemas were the single biggest issue (next to bad Indexes
5 0.9278394 393 high scalability-2008-09-25-GridGain: One Compute Grid, Many Data Grids
Introduction: GridGain was kind enough to present at the September 17th instance of the Silicon Valley Cloud Computing Group . I've been curious about GridGain so I was glad to see them there. In short GridGain is: an open source computational grid framework that enables Java developers to improve general performance of processing intensive applications by splitting and parallelizing the workload . GridGain can also be thought of as a set of middleware primitives for building applications. GridGain's peer group of competitors includes GigaSpaces, Terracotta, Coherence, and Hadoop. The speaker for GridGain was the President and Founder, Nikita Ivanov. He has a very pleasant down-to-earth way about him that contrasts nicely with a field given to religious discussions of complex taxomic definitions. Nikita first talked about cloud computing in general. He feels Java is the perfect gateway for cloud computing. Which is good because GridGain only works with Java . The Java centricity of G
6 0.91416538 873 high scalability-2010-08-06-Hot Scalability Links for Aug 6, 2010
8 0.8911218 1557 high scalability-2013-12-02-Evolution of Bazaarvoice’s Architecture to 500M Unique Users Per Month
9 0.88897634 840 high scalability-2010-06-10-The Four Meta Secrets of Scaling at Facebook
10 0.88739568 1093 high scalability-2011-08-05-Stuff The Internet Says On Scalability For August 5, 2011
11 0.88724339 1514 high scalability-2013-09-09-Need Help with Database Scalability? Understand I-O
12 0.88652259 1606 high scalability-2014-03-05-10 Things You Should Know About Running MongoDB at Scale
13 0.88644451 906 high scalability-2010-09-22-Applying Scalability Patterns to Infrastructure Architecture
14 0.88621354 853 high scalability-2010-07-08-Cloud AWS Infrastructure vs. Physical Infrastructure
15 0.88598382 1197 high scalability-2012-02-21-Pixable Architecture - Crawling, Analyzing, and Ranking 20 Million Photos a Day
16 0.8853386 881 high scalability-2010-08-16-Scaling an AWS infrastructure - Tools and Patterns
17 0.88454401 1180 high scalability-2012-01-24-The State of NoSQL in 2012
18 0.88411641 691 high scalability-2009-08-31-Squarespace Architecture - A Grid Handles Hundreds of Millions of Requests a Month
19 0.88343847 233 high scalability-2008-01-30-How Rackspace Now Uses MapReduce and Hadoop to Query Terabytes of Data
20 0.88304549 70 high scalability-2007-08-22-How many machines do you need to run your site?