high_scalability high_scalability-2007 high_scalability-2007-18 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: MySQL Scale-Out by application partitioning by Oli Sennhauser Eventually every database system hit its limits. Especially on the Internet, where you have millions of users which theoretically access your database simultaneously, eventually your IO system will be a bottleneck. [A] promising but more complex solution with nearly no scale-out limits is application partitioning. If and when you get into the top-1000 rank on alexa [1], you have to think about such solutions. A Quick Hit of What's Inside Horizontal application partitioning, Vertical application partitioning, Disk IO calculations, How to partition an entity
sentIndex sentText sentNum sentScore
1 MySQL Scale-Out by application partitioning by Oli Sennhauser Eventually every database system hit its limits. [sent-1, score-0.953]
2 Especially on the Internet, where you have millions of users which theoretically access your database simultaneously, eventually your IO system will be a bottleneck. [sent-2, score-0.806]
3 [A] promising but more complex solution with nearly no scale-out limits is application partitioning. [sent-3, score-0.858]
4 If and when you get into the top-1000 rank on alexa [1], you have to think about such solutions. [sent-4, score-0.648]
wordName wordTfidf (topN-words)
[('partitioningby', 0.36), ('alexa', 0.293), ('partitioning', 0.288), ('io', 0.276), ('theoretically', 0.264), ('rank', 0.249), ('hit', 0.248), ('promising', 0.208), ('entity', 0.198), ('application', 0.198), ('calculations', 0.197), ('vertical', 0.193), ('simultaneously', 0.192), ('partition', 0.152), ('limits', 0.151), ('quick', 0.139), ('eventually', 0.137), ('nearly', 0.135), ('especially', 0.112), ('millions', 0.105), ('internet', 0.099), ('disk', 0.092), ('database', 0.088), ('complex', 0.087), ('solution', 0.079), ('mysql', 0.079), ('access', 0.078), ('system', 0.073), ('think', 0.063), ('users', 0.061), ('every', 0.058), ('get', 0.043)]
simIndex simValue blogId blogTitle
same-blog 1 1.0 18 high scalability-2007-07-16-Paper: MySQL Scale-Out by application partitioning
Introduction: MySQL Scale-Out by application partitioning by Oli Sennhauser Eventually every database system hit its limits. Especially on the Internet, where you have millions of users which theoretically access your database simultaneously, eventually your IO system will be a bottleneck. [A] promising but more complex solution with nearly no scale-out limits is application partitioning. If and when you get into the top-1000 rank on alexa [1], you have to think about such solutions. A Quick Hit of What's Inside Horizontal application partitioning, Vertical application partitioning, Disk IO calculations, How to partition an entity
2 0.16605395 1646 high scalability-2014-05-12-4 Architecture Issues When Scaling Web Applications: Bottlenecks, Database, CPU, IO
Introduction: This is a guest repost by Venkatesh CM at Architecture Issues Scaling Web Applications . I will cover architecture issues that show up while scaling and performance tuning large scale web application in this blog. Lets start by defining few terms to create common understanding and vocabulary. Later on I will go through different issues that pop-up while scaling web application like Architecture bottlenecks Scaling Database CPU Bound Application IO Bound Application Determining optimal thread pool size of an web application will be covered in next blog. Performance Term performance of web application is used to mean several things. Most developers are primarily concerned with are response time and scalability. Response Time Is the time taken by web application to process request and return response. Applications should respond to requests (response time) within acceptable duration. If application is taking beyond the acceptable time, it is said to
3 0.163634 5 high scalability-2007-07-10-mixi.jp Architecture
Introduction: Mixi is a fast growing social networking site in Japan. They provide services like: diary, community, message, review, and photo album. Having a lot in common with LiveJournal they also developed many of the same approaches. Their write up on how they scaled their system is easily one of the best out there. Site: http://mixi.jp Information Sources mixi.jp - scaling out with open source Platform Linux Apache MySQL Perl Memcached Squid Shard What's Inside? They grew to approximately 4 million users in two years and add over 15,000 new users/day. Ranks 35th on Alexa and 3rd in Japan. More than 100 MySQL servers Add more than 10 servers/month Use non-persistent connections. Diary traffic is 85% read and 15% write. Message traffic is is 75% read and 25% write. Ran into replication performance problems so they had to split the database. Considered splitting vertically by user or splitting horizontally by table type. The ende
4 0.16118284 511 high scalability-2009-02-12-MySpace Architecture
Introduction: Update: Presentation: Behind the Scenes at MySpace.com . Dan Farino, Chief Systems Architect at MySpace shares details of some of MySpace's cool internal operations tools. MySpace.com is one of the fastest growing site on the Internet with 65 million subscribers and 260,000 new users registering each day. Often criticized for poor performance, MySpace has had to tackle scalability issues few other sites have faced. How did they do it? Site: http://myspace.com Information Sources Presentation: Behind the Scenes at MySpace.com Inside MySpace.com Platform ASP.NET 2.0 Windows IIS SQL Server What's Inside? 300 million users. Pushes 100 gigabits/second to the internet. 10Gb/sec is HTML content. 4,500+ web servers windows 2003/IIS 6.0/APS.NET. 1,200+ cache servers running 64-bit Windows 2003. 16GB of objects cached in RAM. 500+ database servers running 64-bit Windows and SQL Server 2005. MySpace processes 1.5 Billion page views per day and
5 0.155394 88 high scalability-2007-09-10-Blog: Scalable Web Architectures by Royans Tharakan
Introduction: Royans' scalability blog and his main blog are excellent sources of scalability information. Take a look. A Quick Hit of What's Inside Sharding: Different from Partitioning and Federation ? , Adventures of scaling eins.de , Session, state and scalability Site: http://www.royans.net/
6 0.15357387 642 high scalability-2009-06-29-HighScalability Rated #3 Blog for Developers
8 0.11822151 392 high scalability-2008-09-24-Building a Scalable Architecture for Web Apps
9 0.1068328 15 high scalability-2007-07-16-Blog: MySQL Performance Blog - Everything about MySQL Performance.
10 0.099419422 327 high scalability-2008-05-27-How I Learned to Stop Worrying and Love Using a Lot of Disk Space to Scale
11 0.09920492 367 high scalability-2008-08-17-Strategy: Drop Memcached, Add More MySQL Servers
12 0.098107331 625 high scalability-2009-06-10-Managing cross partition transactions in a distributed KV system
13 0.097520687 709 high scalability-2009-09-19-Space Based Programming in .NET
14 0.096425362 972 high scalability-2011-01-11-Google Megastore - 3 Billion Writes and 20 Billion Read Transactions Daily
15 0.095870838 394 high scalability-2008-09-25-HighScalability.com Rated 16th Best Blog for Development Managers
16 0.095701069 637 high scalability-2009-06-24-Habits of Highly Scalable Web Applications
17 0.094776899 252 high scalability-2008-02-18-limit on the number of databases open
18 0.094153047 17 high scalability-2007-07-16-Paper: Guide to Cost-effective Database Scale-Out using MySQL
19 0.094065294 1291 high scalability-2012-07-25-Vertical Scaling Ascendant - How are SSDs Changing Architectures?
20 0.089878909 512 high scalability-2009-02-14-Scaling Digg and Other Web Applications
topicId topicWeight
[(0, 0.1), (1, 0.051), (2, -0.033), (3, -0.045), (4, 0.03), (5, 0.075), (6, -0.033), (7, -0.058), (8, -0.025), (9, -0.023), (10, -0.003), (11, 0.011), (12, -0.026), (13, 0.076), (14, 0.014), (15, -0.018), (16, -0.015), (17, -0.002), (18, 0.034), (19, 0.017), (20, 0.028), (21, 0.074), (22, -0.032), (23, -0.011), (24, 0.034), (25, 0.066), (26, 0.017), (27, -0.049), (28, 0.03), (29, 0.043), (30, 0.029), (31, -0.086), (32, 0.035), (33, 0.066), (34, -0.009), (35, -0.018), (36, -0.061), (37, -0.03), (38, 0.011), (39, -0.004), (40, 0.038), (41, -0.011), (42, -0.019), (43, -0.015), (44, 0.008), (45, -0.003), (46, 0.052), (47, -0.019), (48, -0.123), (49, 0.032)]
simIndex simValue blogId blogTitle
same-blog 1 0.94912612 18 high scalability-2007-07-16-Paper: MySQL Scale-Out by application partitioning
Introduction: MySQL Scale-Out by application partitioning by Oli Sennhauser Eventually every database system hit its limits. Especially on the Internet, where you have millions of users which theoretically access your database simultaneously, eventually your IO system will be a bottleneck. [A] promising but more complex solution with nearly no scale-out limits is application partitioning. If and when you get into the top-1000 rank on alexa [1], you have to think about such solutions. A Quick Hit of What's Inside Horizontal application partitioning, Vertical application partitioning, Disk IO calculations, How to partition an entity
Introduction: Update 2: A HSCALE benchmark finds HSCALE "adds a maximum overhead of about 0.24 ms per query (against a partitioned table)." Future releases promise much improved results. Update: A new presentation at An Introduction to HSCALE . After writing Skype Plans for PostgreSQL to Scale to 1 Billion Users , which shows how Skype smartly uses a proxy architecture for scaling, I'm now seeing MySQL Proxy articles all over the place. It's like those "get rich quick" books that say all you have to do is visualize a giraffe with a big yellow dot superimposed over it and by sympathetic magic giraffes will suddenly stampede into your life. Without realizing it I must have visualized transparent proxies smothered in yellow dots. One of the brightest images is a wonderful series of articles by Peter Romianowski describing the evolution of their proxy architecture. Their application is an OLTP system executing 200 million transaction per month, tables with more than 1.5 billion rows, and a 6
3 0.66772777 1646 high scalability-2014-05-12-4 Architecture Issues When Scaling Web Applications: Bottlenecks, Database, CPU, IO
Introduction: This is a guest repost by Venkatesh CM at Architecture Issues Scaling Web Applications . I will cover architecture issues that show up while scaling and performance tuning large scale web application in this blog. Lets start by defining few terms to create common understanding and vocabulary. Later on I will go through different issues that pop-up while scaling web application like Architecture bottlenecks Scaling Database CPU Bound Application IO Bound Application Determining optimal thread pool size of an web application will be covered in next blog. Performance Term performance of web application is used to mean several things. Most developers are primarily concerned with are response time and scalability. Response Time Is the time taken by web application to process request and return response. Applications should respond to requests (response time) within acceptable duration. If application is taking beyond the acceptable time, it is said to
4 0.63973635 297 high scalability-2008-04-05-Skype Plans for PostgreSQL to Scale to 1 Billion Users
Introduction: Skype uses PostgreSQL as their backend database . PostgreSQL doesn't get enough run in the database world so I was excited to see how PostgreSQL is used "as the main DB for most of [Skype's] business needs." Their approach is to use a traditional stored procedure interface for accessing data and on top of that layer proxy servers which hash SQL requests to a set of database servers that actually carry out queries. The result is a horizontally partitioned system that they think will scale to handle 1 billion users. Skype's goal is an architecture that can handle 1 billion plus users. This level of scale isn't practically solvable with one really big computer, so our masked superhero horizontal scaling comes to the rescue. Hardware is dual or quad Opterons with SCSI RAID. Followed common database progression: Start with one DB. Add new databases partitioned by functionality. Replicate read-mostly data for better read access. Then horizontally partition data across multiple nod
5 0.63651568 847 high scalability-2010-06-23-Product: dbShards - Share Nothing. Shard Everything.
Introduction: I met the CodeFutures folks, makers of dbShards , at Gluecon . They occupy an interesting niche in the database space, somewhere between NoSQL , which jettisons everything SQL, and high end analytics platforms that completely rewrite the backend while keeping a SQL facade. High concept: I think of dbShards as a sort of commercial OLTP mashup of features from HSCALE (partitioning) + MySQL Proxy (transparent intermediate layer) + Memcached (client side sharding) + Gigaspaces (parallel query) + MySQL (transactions). You may find dbShards interesting if you are looking to keep SQL, need scale out writes and reads, need out of the box parallel query capabilities, and would prefer to use a standard platform like MySQL as a base. To learn more about dbShards I asked Cory Isaacson (CEO and CTO) a few devastatingly difficult questions (not really). Who are you, what is dbShards, and what problem was dbShards created to solve? I’m Cory Isaacson, CEO/CTO of CodeFutures Corp
6 0.62812173 793 high scalability-2010-03-10-Saying Yes to NoSQL; Going Steady with Cassandra at Digg
7 0.62132996 857 high scalability-2010-07-13-DbShards Part Deux - The Internals
8 0.60309786 88 high scalability-2007-09-10-Blog: Scalable Web Architectures by Royans Tharakan
9 0.59852493 684 high scalability-2009-08-18-Real World Web: Performance & Scalability
10 0.58506447 607 high scalability-2009-05-26-Database Optimize patterns
11 0.57602429 515 high scalability-2009-02-19-GIS Application Hosting
12 0.57433826 192 high scalability-2007-12-25-IBMer Says LAMP Can't Scale
13 0.57178384 455 high scalability-2008-12-01-MySQL Database Scale-out and Replication for High Growth Businesses
14 0.56930256 137 high scalability-2007-10-30-Database parallelism choices greatly impact scalability
15 0.56905466 345 high scalability-2008-06-11-Pyshards aspires to build sharding toolkit for Python
16 0.56407398 392 high scalability-2008-09-24-Building a Scalable Architecture for Web Apps
17 0.56300086 729 high scalability-2009-10-28-And the winner is: MySQL or Memcached or Tokyo Tyrant?
18 0.56107134 89 high scalability-2007-09-10-Is there a difference between partitioning and federation and sharding?
19 0.55447203 1025 high scalability-2011-04-16-The NewSQL Market Breakdown
20 0.5524658 782 high scalability-2010-02-23-When to migrate your database?
topicId topicWeight
[(1, 0.137), (2, 0.227), (9, 0.342), (79, 0.135)]
simIndex simValue blogId blogTitle
1 0.91477984 604 high scalability-2009-05-20-Paper: Flux: An Adaptive Partitioning Operator for Continuous Query Systems
Introduction: At the core of the new real-time web, which is really really old, are continuous queries. I like how this paper proposed to handle dynamic demand and dynamic resource availability by making the underlying system adaptable, which seems like a very cloudy kind of thing to do. Abstract: The long-running nature of continuous queries poses new scalability challenges for dataflow processing. CQ systems execute pipelined dataflows that may be shared across multiple queries. The scalability of these dataflows is limited by their constituent, stateful operators – e.g. windowed joins or grouping operators. To scale such operators, a natural solution is to partition them across a shared-nothing platform. But in the CQ context, traditional, static techniques for partitioned parallelism can exhibit detrimental imbalances as workload and runtime conditions evolve. Longrunning CQ dataflows must continue to function robustly in the face of these imbalances. To address this challenge, we introduce
2 0.80969477 1462 high scalability-2013-05-22-Strategy: Stop Using Linked-Lists
Introduction: What data structure is more sacred than the link list? If we get rid of it what silly interview questions would we use instead? But not using linked-lists is exactly what Aater Suleman recommends in Should you ever use Linked-Lists? In The Secret To 10 Million Concurrent Connections one of the important strategies is not scribbling data all over memory via pointers because following pointers increases cache misses which reduces performance . And there’s nothing more iconic of pointers than the link list. Here are Aeter's reasons to be anti-linked-list: They reduce the benefit of out-of-order execution. They throw off hardware prefetching. They reduce DRAM and TLB locality. They cannot leverage SIMD. They are harder to send to GPUs. He also demolishes the pros of linked-lists, finding arrays a better option in almost every case. Good discussion in the comment section as not everyone agrees. Patrick Wyatt details how a linked-list threading bug repeated
same-blog 3 0.75499773 18 high scalability-2007-07-16-Paper: MySQL Scale-Out by application partitioning
Introduction: MySQL Scale-Out by application partitioning by Oli Sennhauser Eventually every database system hit its limits. Especially on the Internet, where you have millions of users which theoretically access your database simultaneously, eventually your IO system will be a bottleneck. [A] promising but more complex solution with nearly no scale-out limits is application partitioning. If and when you get into the top-1000 rank on alexa [1], you have to think about such solutions. A Quick Hit of What's Inside Horizontal application partitioning, Vertical application partitioning, Disk IO calculations, How to partition an entity
4 0.73163009 520 high scalability-2009-02-25-Advanced BPM program in USA and India discount for Group Membership
Introduction: One day, Advanced BPM Certified program led by Global Leader, Steve Towers. Latest Case Studies and innovations - hands-on, practical. Event locations USA San Francisco 16 Mar 09 Atlanta 17 Mar 09 New York 19 Mar 09 Chicago 20 Mar 09 www.BESTBPMTRAINING.COM India Mumbai 23 Mar 09 Bangalore 24 Mar 09 Hyderabad 26 Mar 09 Delhi 27 Mar 09 www.BPMTRAININGNOW.COM For more information please visit For registrations, group discounts or further details please contact Caroline.smith@icmgworld.com
5 0.72826201 519 high scalability-2009-02-23-Database Sharding at Netlog, with MySQL and PHP
Introduction: Jurriaan Persyn is a Lead Web Developer at Netlog, a social portal site that gets 50 million unique visitors and 5+ billion page views per month. In this paper Jurriaan goes into a lot of excellent nuts and bolts details about how they used sharding to scale their system. If you are pondering sharding as a solution to your scaling problems you'll want to read this paper. As the paper is quite well organized there's no reason to write a summary, but I especially liked this part from the conclusion: If you can do with simpler solutions (better hardware, more hardware, server tweaking and tuning, vertical partitioning, sql query optimization, ...) that require less development cost, why invest lots of effort in sharding? On the other hand, when your visitor statistics really start blowing through the roof, it is a good direction to go. After all, it worked for us.
6 0.71870798 1270 high scalability-2012-06-22-Stuff The Internet Says On Scalability For June 22, 2012
7 0.71745062 577 high scalability-2009-04-22-Gear6 Web cache - the hardware solution for working with Memcache
9 0.68040025 169 high scalability-2007-12-01-many website, one setup, many databases
10 0.67447215 409 high scalability-2008-10-13-Challenges from large scale computing at Google
11 0.64666063 15 high scalability-2007-07-16-Blog: MySQL Performance Blog - Everything about MySQL Performance.
12 0.64425176 63 high scalability-2007-08-09-Lots of questions for high scalability - high availability
13 0.64416081 343 high scalability-2008-06-09-Apple's iPhone to Use a Centralized Push Based Notification Architecture
14 0.64365864 942 high scalability-2010-11-15-Strategy: Biggest Performance Impact is to Reduce the Number of HTTP Requests
15 0.64316785 719 high scalability-2009-10-09-Have you collectl'd yet? If not, maybe collectl-utils will make it easier to do so
16 0.64186174 415 high scalability-2008-10-15-Need help with your Hadoop deployment? This company may help!
17 0.64118075 946 high scalability-2010-11-22-Strategy: Google Sends Canary Requests into the Data Mine
18 0.64067191 1276 high scalability-2012-07-04-Top Features of a Scalable Database
19 0.63951385 636 high scalability-2009-06-23-Learn How to Exploit Multiple Cores for Better Performance and Scalability
20 0.63696551 900 high scalability-2010-09-11-Google's Colossus Makes Search Real-time by Dumping MapReduce