high_scalability high_scalability-2008 high_scalability-2008-473 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Update: Presentation: Second Life’s Architecture . Ian Wilkes, VP of Systems Engineering, describes the architecture used by the popular game named Second Life. Ian presents how the architecture was at its debut and how it evolved over years as users and features have been added. Second Life is a 3-D virtual world created by its Residents. Virtual Worlds are expected to be more and more popular on the internet so their architecture might be of interest. Especially important is the appearance of open virtual worlds or metaverses. What happens when video games meet Web 2.0? What happens is the metaverse . Information Sources Second Life runs MySQL Interview with Ian Wilkes TechTrends: Inside Linden Lab Town Hall with Cory Linden InformationWeek articles ( 1 , 2 ) and blog Second Life Wiki: Server Architecture Wikipedia: Second Life Server Second Life Blog Second Life: A Guide to Your Virtual World Platform
sentIndex sentText sentNum sentScore
1 Ian presents how the architecture was at its debut and how it evolved over years as users and features have been added. [sent-3, score-0.086]
2 Second Life is a 3-D virtual world created by its Residents. [sent-4, score-0.116]
3 Especially important is the appearance of open virtual worlds or metaverses. [sent-6, score-0.299]
4 The Stats ~1M active users ~95M user hours per quarter ~70K peak concurrent users (40% annual growth) ~12Gbit/sec aggregate bandwidth (in 2007) Staff (in 2006) 70 FTE + 20 part time "about 22 are programmers working on SL itself. [sent-11, score-0.064]
5 At any one time probably 1/3 of the team is on infrastructure, 1/3 is on new features and 1/3 is on various maintenance tasks (bug fixes, general stability and speed improvements) or improvements to existing features. [sent-12, score-0.079]
6 " And each sim runs on a separate core of a server. [sent-15, score-0.3]
7 The Simulator is the primary SL C++ server process which runs on most servers. [sent-16, score-0.111]
8 As the viewer moves through the world it is handled off from one simulator to another. [sent-17, score-0.315]
9 If it can't keep up, it will attempt time dialation without reducing frame rate. [sent-19, score-0.187]
10 ) Runs Linden Scripting Language (LSL) scripts Scripting has been recently upgraded to the much faster Mono scripting engine Handles chat and instant messages One big clustered filesystem ~100TB Stores asset data such as textures. [sent-21, score-0.099]
11 It achieves high scalability by using non-blocking io while at the same time retaining high programmer usability by using coroutines to make the non-blocking io operations appear blocking at the source code level. [sent-23, score-0.281]
12 Mulib is a REST web service framework built on top of eventlet 2000+ Servers in 2007 ~6000 Servers in early 2008 Plans to upgrade to ~10000 (? [sent-24, score-0.065]
13 Class 5 is on pure 1Gb Asset Server MySQL database Second Life has started with One Database, and have subsequently been forced into clustering. [sent-26, score-0.077]
14 Rather than attempt to build the one, impossibly large database – all hail the Central Database – or one, impossibly large central cluster – all hail the Cluster – Linden Lab instead adopted a divide and conquer strategy based around data partitioning. [sent-28, score-0.892]
15 The good thing is that UUIDs– 128-bit unique identifiers – are associated with most things in Second Life, so partitioning is generally doable. [sent-29, score-0.068]
16 Certain services has been moved off of MySQL – or cached (Squid) between the queries and MySQL. [sent-31, score-0.069]
17 It proved to be easier to scale, develop and maintain than many of their older technologies, and as a result, it plays an increasingly important role in the Second Life platform as Linden Lab migrates their legacy code to web services. [sent-33, score-0.07]
wordName wordTfidf (topN-words)
[('life', 0.299), ('sl', 0.283), ('simulator', 0.241), ('collision', 0.209), ('sim', 0.189), ('lab', 0.177), ('hail', 0.171), ('linden', 0.154), ('impossibly', 0.153), ('second', 0.139), ('class', 0.137), ('ian', 0.122), ('virtual', 0.116), ('runs', 0.111), ('worlds', 0.106), ('attempt', 0.101), ('scripting', 0.099), ('presence', 0.091), ('locations', 0.086), ('hardwaredo', 0.086), ('blogsecond', 0.086), ('debut', 0.086), ('dialation', 0.086), ('hall', 0.086), ('instantiation', 0.086), ('terrain', 0.086), ('uplinks', 0.086), ('velocities', 0.086), ('improvements', 0.079), ('appearance', 0.077), ('coroutines', 0.077), ('cory', 0.077), ('subsequently', 0.077), ('central', 0.075), ('viewer', 0.074), ('prioritized', 0.074), ('retaining', 0.072), ('migrates', 0.07), ('moved', 0.069), ('viewers', 0.068), ('conquer', 0.068), ('identifiers', 0.068), ('varies', 0.068), ('track', 0.068), ('opteron', 0.066), ('meter', 0.066), ('io', 0.066), ('upgrade', 0.065), ('quarter', 0.064), ('inside', 0.063)]
simIndex simValue blogId blogTitle
same-blog 1 1.0000002 473 high scalability-2008-12-20-Second Life Architecture - The Grid
Introduction: Update: Presentation: Second Life’s Architecture . Ian Wilkes, VP of Systems Engineering, describes the architecture used by the popular game named Second Life. Ian presents how the architecture was at its debut and how it evolved over years as users and features have been added. Second Life is a 3-D virtual world created by its Residents. Virtual Worlds are expected to be more and more popular on the internet so their architecture might be of interest. Especially important is the appearance of open virtual worlds or metaverses. What happens when video games meet Web 2.0? What happens is the metaverse . Information Sources Second Life runs MySQL Interview with Ian Wilkes TechTrends: Inside Linden Lab Town Hall with Cory Linden InformationWeek articles ( 1 , 2 ) and blog Second Life Wiki: Server Architecture Wikipedia: Second Life Server Second Life Blog Second Life: A Guide to Your Virtual World Platform
2 0.10629883 21 high scalability-2007-07-23-GoogleTalk Architecture
Introduction: Google Talk is Google's instant communications service. Interestingly the IM messages aren't the major architectural challenge, handling user presence indications dominate the design. They also have the challenge of handling small low latency messages and integrating with many other systems. How do they do it? Site: http://www.google.com/talk Information Sources GoogleTalk Architecture Platform Linux Java Google Stack Shard What's Inside? The Stats Support presence and messages for millions of users. Handles billions of packets per day in under 100ms. IM is different than many other applications because the requests are small packets. Routing and application logic are applied per packet for sender and receiver. Messages must be delivered in-order. Architecture extends to new clients and Google services. Lessons Learned Measure the right thing. - People ask about how many IMs do you deliver or how many active users. Turns out not
3 0.10606599 796 high scalability-2010-03-16-Justin.tv's Live Video Broadcasting Architecture
Introduction: The future is live. The future is real-time. The future is now. That's the hype anyway. And as it has a habit of doing, the hype is slowly becoming reality. We are seeing live searches, live tweets, live location, live reality augmentation, live crab (fresh and local), and live event publishing. One of the most challenging of all live technologies is that of live video broadcasting. Imagine a world in which everyone becomes a broadcaster and a consumer of video streams, all in real-time (< 250 msec latency), all so you can talk and interact directly without feeling like you are in the middle of a time shift war. The resources and the engineering needed to make this happened must be substantial. How do you do that? To find out I talked to Kyle Vogt, Justin.tv Founder and VP of Engineering. Justin.tv certainly has the numbers. Their 30 million unique monthly visitors even outshine YouTube in the video upload game, reportedly uploading nearly 30 hours per minute of video compared to Y
4 0.10553161 1359 high scalability-2012-11-15-Gone Fishin': Justin.Tv's Live Video Broadcasting Architecture
Introduction: This is one of my favorite posts for a couple of reasons. I think it gives a lot of useful information in an interesting space. And Kyle Vogt was just a real pleasure to talk to. He was very helpful and forthcoming, which makes the whole experience better for everyone. The future is live. The future is real-time. The future is now. That's the hype anyway. And as it has a habit of doing, the hype is slowly becoming reality. We are seeing live searches, live tweets, live location, live reality augmentation, live crab (fresh and local), and live event publishing. One of the most challenging of all live technologies is that of live video broadcasting. Imagine a world in which everyone becomes a broadcaster and a consumer of video streams, all in real-time (< 250 msec latency), all so you can talk and interact directly without feeling like you are in the middle of a time shift war. The resources and the engineering needed to make this happened must be substantial. How do you do tha
5 0.10463824 496 high scalability-2009-01-17-Scaling in Games & Virtual Worlds
Introduction: "Online games and virtual worlds have familiar scaling requirements, but don’t be fooled: everything you know is wrong." Jim Waldo, Sun Microsystems Laboratories * The computational environment for online games or virtual worlds is close to the exact inverse of that found in most markets serviced by the high-tech industry. * The need for a heavyweight client is, in part, an outcome of the evolution of these games. * Latency is the enemy of fun—and therefore the enemy of online games and virtual worlds. * The game server is used both to discourage cheating (by making it much more difficult) and to detect cheating (by seeing patterns of divergence between the game state reported by the client and the game state held by the server). Peer-to-peer technologies might seem a natural fit for the first role of the game server, but this second role means that few if any games or worlds trust their peers enough to avoid the server component. * Using multiple servers is a basic mecha
6 0.10192198 448 high scalability-2008-11-22-Google Architecture
7 0.098738916 808 high scalability-2010-04-12-Poppen.de Architecture
8 0.096475422 274 high scalability-2008-03-12-YouTube Architecture
9 0.096236274 538 high scalability-2009-03-16-Are Cloud Based Memory Architectures the Next Big Thing?
10 0.096181251 1089 high scalability-2011-07-29-Stuff The Internet Says On Scalability For July 29, 2011
11 0.096051946 1210 high scalability-2012-03-16-Stuff The Internet Says On Scalability For March 16, 2012
12 0.094215266 750 high scalability-2009-12-16-Building Super Scalable Systems: Blade Runner Meets Autonomic Computing in the Ambient Cloud
14 0.092746831 726 high scalability-2009-10-22-Paper: The Case for RAMClouds: Scalable High-Performance Storage Entirely in DRAM
15 0.091446169 1240 high scalability-2012-05-07-Startups are Creating a New System of the World for IT
16 0.089871712 511 high scalability-2009-02-12-MySpace Architecture
17 0.088216759 72 high scalability-2007-08-22-Wikimedia architecture
18 0.087518059 904 high scalability-2010-09-21-Playfish's Social Gaming Architecture - 50 Million Monthly Users and Growing
19 0.086354829 1501 high scalability-2013-08-13-In Memoriam: Lavabit Architecture - Creating a Scalable Email Service
20 0.08420261 954 high scalability-2010-12-06-What the heck are you actually using NoSQL for?
topicId topicWeight
[(0, 0.181), (1, 0.07), (2, 0.009), (3, -0.037), (4, -0.0), (5, -0.004), (6, -0.005), (7, -0.009), (8, -0.007), (9, 0.036), (10, 0.009), (11, 0.014), (12, 0.032), (13, 0.03), (14, 0.012), (15, 0.042), (16, 0.003), (17, 0.011), (18, -0.014), (19, -0.015), (20, 0.027), (21, 0.005), (22, -0.018), (23, -0.033), (24, 0.013), (25, 0.025), (26, -0.001), (27, -0.011), (28, -0.02), (29, 0.019), (30, 0.015), (31, 0.011), (32, -0.063), (33, -0.021), (34, 0.019), (35, -0.019), (36, 0.003), (37, -0.033), (38, 0.056), (39, 0.021), (40, 0.006), (41, -0.067), (42, 0.059), (43, -0.001), (44, 0.003), (45, -0.013), (46, 0.048), (47, -0.026), (48, -0.047), (49, 0.03)]
simIndex simValue blogId blogTitle
same-blog 1 0.95893407 473 high scalability-2008-12-20-Second Life Architecture - The Grid
Introduction: Update: Presentation: Second Life’s Architecture . Ian Wilkes, VP of Systems Engineering, describes the architecture used by the popular game named Second Life. Ian presents how the architecture was at its debut and how it evolved over years as users and features have been added. Second Life is a 3-D virtual world created by its Residents. Virtual Worlds are expected to be more and more popular on the internet so their architecture might be of interest. Especially important is the appearance of open virtual worlds or metaverses. What happens when video games meet Web 2.0? What happens is the metaverse . Information Sources Second Life runs MySQL Interview with Ian Wilkes TechTrends: Inside Linden Lab Town Hall with Cory Linden InformationWeek articles ( 1 , 2 ) and blog Second Life Wiki: Server Architecture Wikipedia: Second Life Server Second Life Blog Second Life: A Guide to Your Virtual World Platform
2 0.74993861 1645 high scalability-2014-05-09-Stuff The Internet Says On Scalability For May 9th, 2014
Introduction: Hey, it's HighScalability time: NASA captures Guatemala volcano erupting from space 40,000 exabytes : from now until 2020, the digital universe will about double every two years; $650,000 : amount raised by the MaydayPAC in one week. Quotable Quotes: @BenedictEvans : Masayoshi Son: $20m initial investment in Alibaba, current stake worth $58bn. @iamdevloper : I sneezed earlier and Siri compiled it to valid Perl. @cdixon : "There is not enough competition in the last mile market to allow a true market to function" @PatrickMcFadin : Get ready for some serious server density. AMD is working on K12, brand-new x86 and ARM cores. This plus 8T SSD? With age comes changing priorities. Facebook is now 10 and has grown up . They are no longer moving fast and breaking things. They are now into the stability thing. Letting developers know they are a stable platform. The play is to get all that beautiful data from developers by bei
3 0.73792517 1542 high scalability-2013-11-04-ESPN's Architecture at Scale - Operating at 100,000 Duh Nuh Nuhs Per Second
Introduction: ESPN went on airin 1978. In those 30+ years think of the wonders we've seen! When I think of ESPN I think of a world wide brand that is the very definition of prime time. And it shows in their stats. ESPN.com peaks at 100,000 requests per second. Their peak event is, not surprisingly, the World Cup. But would you be surprised to learn ESPN is powered by only a few hundred servers and a couple of dozen engineers? I was.And would you be surprised to learn ESPN is undergoing a fundamental transition from an Enterprise architecture to one capable of handling web scale loads driven by increasing mobile usage, personalization, and a service orientation? Again, thinking ESPN was just about watching sports on TV, I was surprised. ESPN is becoming much more than that. ESPN is becoming a sports platform. How does ESPN handle all of this complexity, responsibility, change, and load? Unlike most every other profile on HighScalability. The fascinating story of ESPN's architecture is told byManny Pe
4 0.73552877 68 high scalability-2007-08-20-TypePad Architecture
Introduction: TypePad is considered the largest paid blogging service in the world. After experience problems because of their meteoric growth, they eventually transitioned to an architecture patterned after their sister company, LiveJournal. Site: http://www.typepad.com/ The Platform MySQL Memcached Perl MogileFS Apache Linux The Stats As of 2005 TypePad sends 250mbps of traffic using multiple network pipes for 3TB of traffic a day. They were growing by 10-20% each month. I was unable to find more recent statistics. The Architecture Original Architecture: - Single server running Linux, Apache, Postgres, Perl, mod_perl - Storage was NFS on a filer. A Devastating Crash Caused a New Direction - A RAID controller failed and spewed data across all RAID disks. - The database was corrupted and the backups were corrupted. - Their redundant filers suffered from "split brain" syndrome. They move to LiveJournal Architecture type architecture which isn't surprising
5 0.73240876 160 high scalability-2007-11-19-Tailrank Architecture - Learn How to Track Memes Across the Entire Blogosphere
Introduction: Ever feel like the blogosphere is 500 million channels with nothing on? Tailrank finds the internet's hottest channels by indexing over 24M weblogs and feeds per hour. That's 52TB of raw blog content (no, not sewage) a month and requires continuously processing 160Mbits of IO. How do they do that? This is an email interview with Kevin Burton, founder and CEO of Tailrank.com. Kevin was kind enough to take the time to explain how they scale to index the entire blogosphere. Sites Tailrank - We track the hottest news in the blogosphere! Spinn3r - A blog spider you can specialize with your own behavior instead of creating your own. Kevin Burton's Blog - his blog is an indexing mix of politics and technical talk. Both are always interesting. Platform MySQL Java Linux (Debian) Apache Squid PowerDNS DAS storage. Federated database. ServerBeach hosting. Job scheduling system for work distribution. Interview What is your system is f
6 0.72813237 1261 high scalability-2012-06-08-Stuff The Internet Says On Scalability For June 8, 2012
7 0.72690099 1516 high scalability-2013-09-13-Stuff The Internet Says On Scalability For September 13, 2013
8 0.72424436 511 high scalability-2009-02-12-MySpace Architecture
9 0.72419542 1166 high scalability-2011-12-30-Stuff The Internet Says On Scalability For December 30, 2011
10 0.72002071 513 high scalability-2009-02-16-Handle 1 Billion Events Per Day Using a Memory Grid
11 0.71679187 297 high scalability-2008-04-05-Skype Plans for PostgreSQL to Scale to 1 Billion Users
12 0.71392399 1573 high scalability-2014-01-06-How HipChat Stores and Indexes Billions of Messages Using ElasticSearch and Redis
15 0.70953888 1501 high scalability-2013-08-13-In Memoriam: Lavabit Architecture - Creating a Scalable Email Service
16 0.70930725 1315 high scalability-2012-08-30-Stuff The Internet Says On Scalability For August 31, 2012
17 0.69789881 1154 high scalability-2011-12-09-Stuff The Internet Says On Scalability For December 9, 2011
18 0.69747204 1068 high scalability-2011-06-27-TripAdvisor Architecture - 40M Visitors, 200M Dynamic Page Views, 30TB Data
19 0.69711429 1046 high scalability-2011-05-23-Evernote Architecture - 9 Million Users and 150 Million Requests a Day
20 0.69644004 5 high scalability-2007-07-10-mixi.jp Architecture
topicId topicWeight
[(1, 0.128), (2, 0.157), (10, 0.06), (30, 0.038), (42, 0.235), (61, 0.124), (79, 0.091), (85, 0.036), (94, 0.034)]
simIndex simValue blogId blogTitle
1 0.93795747 1608 high scalability-2014-03-10-Let's Play a Game of Take It or Leave It - Game 1
Introduction: The way this game is played is you read a few statements on some hot topics below. If you agree with a statement then you “take it”; if you disagree then you “leave it.” And if you are so moved please write a convincing comment as to why. Got it? Snowden vs. the State. Snowden represents true the spirit of freedom and is not a threat to all we hold dear. Walled Garden vs. Federated Freedom. The Walled Garden has won the last decade. The cycle of life will return the balance and federated services will once again win the day. Mobile + messaging vs. Le Web. Mobile + messaging is eating search and the web, changing the way things are found, discovered, and bought. Fiat vs. Cryptocurrency. BitCoin has had its 400 million dollars of fame, it’s on the way out, a tulip gone out of bloom. True Detective vs. The Field. True Detective is the best show on TV, ever. Wired and Breaking Bad need not apply.
2 0.87065887 1227 high scalability-2012-04-13-Stuff The Internet Says On Scalability For April 13, 2012
Introduction: It's HighScalability Time: 50 million in 50 days : Draw Something downloads; 40 million concurrent users : Skype Key to making sensors ubiquitous is getting the BOM cost down. Here's a dream way of making that happen: Bye-Bye Batteries: Radio Waves as a Low-Power Source . “Silicon technology has advanced to the point where even tiny amounts of energy can do useful work.” No batteries == cheaper, smaller products == ubiquity. The MySQL “swap insanity” problem and the effects of the NUMA architecture . Jeremy Cole with a spectacular article on the differences between NUMA and SMP/UMA systems and the mostly unsatisfactory tricks required to get MySQL to perform on NUMA systems. There are really two issues: the evils of an OS controlled swap and NUMA performance effects due to a single node (in the NUMA sense) running out of memory. This is the kind of stuff you only see when you push your systems to the edge. Also, Measuring NUMA effects with the STREAM benchmark , MongoDB
same-blog 3 0.86383539 473 high scalability-2008-12-20-Second Life Architecture - The Grid
Introduction: Update: Presentation: Second Life’s Architecture . Ian Wilkes, VP of Systems Engineering, describes the architecture used by the popular game named Second Life. Ian presents how the architecture was at its debut and how it evolved over years as users and features have been added. Second Life is a 3-D virtual world created by its Residents. Virtual Worlds are expected to be more and more popular on the internet so their architecture might be of interest. Especially important is the appearance of open virtual worlds or metaverses. What happens when video games meet Web 2.0? What happens is the metaverse . Information Sources Second Life runs MySQL Interview with Ian Wilkes TechTrends: Inside Linden Lab Town Hall with Cory Linden InformationWeek articles ( 1 , 2 ) and blog Second Life Wiki: Server Architecture Wikipedia: Second Life Server Second Life Blog Second Life: A Guide to Your Virtual World Platform
4 0.83129233 1262 high scalability-2012-06-11-Monday Fun: Seven Databases in Song
Introduction: If you understand things best when they're formatted as a musical, this video is for you. It teaches the essentials of PostgreSQL, Riak, HBase, MongoDB, CouchDB, Neo4J and Redis in the style of My Fair Lady. And for a change, it's very SFW.
5 0.82498348 1427 high scalability-2013-03-20-Dart - Is it the Future of the Web?
Introduction: John McCutchan , after a long career spent working on the Linux kernel and being hired out as a code optimization guru, joined Google's Dart team. A curious hire until you watch Bringing SIMD to the Web via Dart , where John makes a programmer accessible explanation of why he likes Dart: performance, performance, performance. Dart is an open-source Web programming language developed by Google . The motivation for Dart is twofold: provide a language capable of scaling up to the complex web applications that are becoming the norm. Think Gmail. And provide a single language capable of working on both the client and server. Towards those ends Dart is a complete language, full tool environment, and provides an advanced Web UI framework for building web applications at a high level of abstraction. Why isn't JavaScript good enough? The fear is for large web apps the nature of JavaScript puts crippling limits on potential performance improvements, which will cause web app
6 0.78744376 572 high scalability-2009-04-16-Paper: The End of an Architectural Era (It’s Time for a Complete Rewrite)
7 0.77548444 1588 high scalability-2014-01-31-Stuff The Internet Says On Scalability For January 31st, 2014
8 0.76698005 591 high scalability-2009-05-06-Dyrad
9 0.76521146 1010 high scalability-2011-03-24-Strategy: Disk Backup for Speed, Tape Backup to Save Your Bacon, Just Ask Google
10 0.74567145 1402 high scalability-2013-02-07-Ask HighScalability: Web asset server concept - 3rd party software available?
11 0.74407715 934 high scalability-2010-11-04-Facebook at 13 Million Queries Per Second Recommends: Minimize Request Variance
12 0.74399716 804 high scalability-2010-04-06-Sponsored Post: Event - Social Developer Summit
13 0.74397391 814 high scalability-2010-04-20-Sponsored Post: Event - Social Developer Summit
14 0.73759937 383 high scalability-2008-09-10-Shard servers -- go big or small?
15 0.73669457 1289 high scalability-2012-07-23-State of the CDN: More Traffic, Stable Prices, More Products, Profits - Not So Much
16 0.73573822 1180 high scalability-2012-01-24-The State of NoSQL in 2012
17 0.73535037 931 high scalability-2010-10-28-Notes from A NOSQL Evening in Palo Alto
18 0.73501521 1506 high scalability-2013-08-23-Stuff The Internet Says On Scalability For August 23, 2013
19 0.73475087 853 high scalability-2010-07-08-Cloud AWS Infrastructure vs. Physical Infrastructure
20 0.73467946 1189 high scalability-2012-02-07-Hypertable Routs HBase in Performance Test -- HBase Overwhelmed by Garbage Collection