high_scalability high_scalability-2007 high_scalability-2007-173 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: I’ve been a part of some late night release procedures and they’re never fun. You’ve got QA, Dev, IT and a handful of managers sitting in their jammies in a group IM (or worse, a conference call) from 2:00 AM until way too early in the morning. Everyone’s grumpy and sleepy, causing the release to be more difficult and take longer. Sometimes the dreaded “rollback!” is yelled. All this because you’re running a high profile website that needs to be accessible 24/7, and 2:00 AM - 5:00 AM downtime is better than daytime downtime. If you're a site that doesn't have 10s of thousands to drop on a real http load balancer, use this strategy to release software during business hours with no downtime using apache's mod_proxy_balancer....
sentIndex sentText sentNum sentScore
1 I’ve been a part of some late night release procedures and they’re never fun. [sent-1, score-0.863]
2 You’ve got QA, Dev, IT and a handful of managers sitting in their jammies in a group IM (or worse, a conference call) from 2:00 AM until way too early in the morning. [sent-2, score-0.835]
3 Everyone’s grumpy and sleepy, causing the release to be more difficult and take longer. [sent-3, score-0.588]
4 All this because you’re running a high profile website that needs to be accessible 24/7, and 2:00 AM - 5:00 AM downtime is better than daytime downtime. [sent-6, score-0.783]
5 If you're a site that doesn't have 10s of thousands to drop on a real http load balancer, use this strategy to release software during business hours with no downtime using apache's mod_proxy_balancer. [sent-7, score-1.321]
wordName wordTfidf (topN-words)
[('ve', 0.43), ('re', 0.336), ('release', 0.305), ('downtime', 0.274), ('dreaded', 0.237), ('rollback', 0.185), ('im', 0.174), ('handful', 0.163), ('procedures', 0.156), ('causing', 0.154), ('qa', 0.152), ('sitting', 0.149), ('late', 0.147), ('dev', 0.143), ('worse', 0.138), ('night', 0.132), ('managers', 0.13), ('accessible', 0.126), ('balancer', 0.125), ('drop', 0.121), ('profile', 0.115), ('sometimes', 0.11), ('conference', 0.098), ('apache', 0.09), ('hours', 0.09), ('difficult', 0.09), ('got', 0.086), ('early', 0.085), ('group', 0.085), ('everyone', 0.085), ('call', 0.084), ('thousands', 0.082), ('strategy', 0.081), ('website', 0.068), ('never', 0.067), ('needs', 0.066), ('business', 0.065), ('http', 0.06), ('part', 0.056), ('site', 0.055), ('real', 0.054), ('running', 0.05), ('better', 0.047), ('software', 0.043), ('load', 0.043), ('way', 0.039), ('take', 0.039), ('high', 0.037), ('using', 0.025), ('use', 0.023)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999994 173 high scalability-2007-12-05-Easier Production Releases
Introduction: I’ve been a part of some late night release procedures and they’re never fun. You’ve got QA, Dev, IT and a handful of managers sitting in their jammies in a group IM (or worse, a conference call) from 2:00 AM until way too early in the morning. Everyone’s grumpy and sleepy, causing the release to be more difficult and take longer. Sometimes the dreaded “rollback!” is yelled. All this because you’re running a high profile website that needs to be accessible 24/7, and 2:00 AM - 5:00 AM downtime is better than daytime downtime. If you're a site that doesn't have 10s of thousands to drop on a real http load balancer, use this strategy to release software during business hours with no downtime using apache's mod_proxy_balancer....
Introduction: At a Cloud Computing Meetup , Siddharth "Sid" Anand of Netflix, backed by a merry band of Netflixians, gave an interesting talk: Keeping Movies Running Amid Thunderstorms . While the talk gave a good overview of their move to the cloud, issues with capacity planning, thundering herds , latency problems, and simian armageddon , I found myself most taken with how they handle software deployment in the cloud . I've worked on half a dozen or more build and deployment systems, some small, some quite large, but never for a large organization like Netflix in the cloud. The cloud has this amazing capability that has never existed before that enables a novel approach to fault-tolerant software deployments: the ability to spin up huge numbers of instances to completely run a new release while running the old release at the same time . The process goes something like: A canary machine is launched first with the new software load running real traffic to sanity test the load in a p
3 0.11370042 1531 high scalability-2013-10-13-AIDA: Badoo’s journey into Continuous Integration
Introduction: It’s hardly news to anyone that product development and testing involve a lot of boring routine work, which can lead to human error. To avoid complications stemming from this, we use AIDA. AIDA (Automated Interactive Deploy Assistant) is a utility that automatically performs many of the processes in Git, TeamCity and JIRA. In this post, we focus on how through using AIDA we were able to automate multiple workflows and create a scheme of continuous integration. We’ll start by looking at the version control system (VCS) we use here at Badoo, specifically how Git is used to automate creation of release branches, and their subsequent merging. Then we’ll discuss AIDA’s major contribution to both JIRA integration and TeamCity. Git flow The Badoo Team uses Git as a version control system. Our model ensures each task is developed and tested in a separate branch. The branch name consists of the ticket number in JIRA and a description of the problem. BFG-9000_All_developers_should
4 0.085360393 378 high scalability-2008-09-03-Some Facebook Secrets to Better Operations
Introduction: Kim Nash in an interview with Jonathan Heiliger , Facebook VP of technical operations, provides some juicy details on how Facebook handles operations. Operations is one of those departments everyone runs differently as it is usually an ontogeny recapitulates phylogeny situation. With 2,000 databases, 25 terabytes of cache, 90 million active users, and 10,000 servers you know Facebook has some serious operational issues. What are some of Facebook's secrets to better operations? Frequent Releases . A major release once a week and a minor releases every few days. Create a Cyber Liability Group . At one time operations was distributed amongst several groups. A permanent operations group was created to isolate problems and revert problem software components back to previously known good states. The ability of a separate team to handle rollbacks speaks to a great deal of standardization and advanced tool building. Distribute Team Across Time Zones . Split the operations team ac
5 0.077031732 1371 high scalability-2012-12-12-Pinterest Cut Costs from $54 to $20 Per Hour by Automatically Shutting Down Systems
Introduction: We've long known one of the virtues of the cloud is, through the magic of services and automation, that systems can be shut or tuned down when not in use. What may be surprising is how much money can be saved. This aspect of cloudiness got a lot of pub at AWS re:Invent and is being rebranded under the term Cost-Aware Architecture . An interesting example was given by Ryan Park, Pinterest’s technical operations lead: 20% of their systems are shutdown after hours in response to traffic loads Reserved instances are used for standard traffic On-demand and spot instances are used to handle the elastic load throughout the day. When more servers are needed for an auto-scaled service, spot requests are opened and on-demand instances are started at the same time. Most services are targeted to run at about 50% on-demand and 50% spot. Watchdog processes continually check what's running. More instances are launched when needed and terminated when not needed. If spot prices spik
6 0.075236619 539 high scalability-2009-03-16-Books: Web 2.0 Architectures and Cloud Application Architectures
7 0.073076054 142 high scalability-2007-11-05-Strategy: Diagonal Scaling - Don't Forget to Scale Out AND Up
8 0.07266748 537 high scalability-2009-03-12-QCon London 2009: Database projects to watch closely
9 0.070852607 1068 high scalability-2011-06-27-TripAdvisor Architecture - 40M Visitors, 200M Dynamic Page Views, 30TB Data
11 0.069072984 516 high scalability-2009-02-19-Heavy upload server scalability
12 0.068138942 1508 high scalability-2013-08-28-Sean Hull's 20 Biggest Bottlenecks that Reduce and Slow Down Scalability
13 0.067450285 679 high scalability-2009-08-11-13 Scalability Best Practices
14 0.066534989 1628 high scalability-2014-04-08-Microservices - Not a free lunch!
16 0.06304802 933 high scalability-2010-11-01-Hot Trend: Move Behavior to Data for a New Interactive Application Architecture
17 0.061465383 189 high scalability-2007-12-21-Strategy: Limit Result Sets
19 0.060193598 627 high scalability-2009-06-11-Yahoo! Distribution of Hadoop
20 0.059282463 1419 high scalability-2013-03-07-It's a VM Wasteland - A Near Optimal Packing of VMs to Machines Reduces TCO by 22%
topicId topicWeight
[(0, 0.085), (1, 0.012), (2, -0.005), (3, -0.034), (4, 0.019), (5, -0.04), (6, -0.003), (7, -0.019), (8, -0.012), (9, -0.022), (10, -0.031), (11, 0.007), (12, 0.03), (13, -0.035), (14, 0.022), (15, -0.007), (16, 0.046), (17, -0.011), (18, 0.026), (19, 0.046), (20, 0.015), (21, 0.003), (22, 0.035), (23, -0.0), (24, -0.019), (25, 0.016), (26, -0.015), (27, 0.019), (28, -0.015), (29, 0.029), (30, -0.005), (31, 0.01), (32, -0.034), (33, -0.005), (34, -0.002), (35, 0.011), (36, -0.043), (37, -0.011), (38, -0.017), (39, -0.01), (40, -0.032), (41, 0.008), (42, 0.031), (43, 0.011), (44, 0.011), (45, 0.024), (46, 0.001), (47, 0.032), (48, -0.003), (49, 0.016)]
simIndex simValue blogId blogTitle
same-blog 1 0.96137786 173 high scalability-2007-12-05-Easier Production Releases
Introduction: I’ve been a part of some late night release procedures and they’re never fun. You’ve got QA, Dev, IT and a handful of managers sitting in their jammies in a group IM (or worse, a conference call) from 2:00 AM until way too early in the morning. Everyone’s grumpy and sleepy, causing the release to be more difficult and take longer. Sometimes the dreaded “rollback!” is yelled. All this because you’re running a high profile website that needs to be accessible 24/7, and 2:00 AM - 5:00 AM downtime is better than daytime downtime. If you're a site that doesn't have 10s of thousands to drop on a real http load balancer, use this strategy to release software during business hours with no downtime using apache's mod_proxy_balancer....
2 0.67950749 1531 high scalability-2013-10-13-AIDA: Badoo’s journey into Continuous Integration
Introduction: It’s hardly news to anyone that product development and testing involve a lot of boring routine work, which can lead to human error. To avoid complications stemming from this, we use AIDA. AIDA (Automated Interactive Deploy Assistant) is a utility that automatically performs many of the processes in Git, TeamCity and JIRA. In this post, we focus on how through using AIDA we were able to automate multiple workflows and create a scheme of continuous integration. We’ll start by looking at the version control system (VCS) we use here at Badoo, specifically how Git is used to automate creation of release branches, and their subsequent merging. Then we’ll discuss AIDA’s major contribution to both JIRA integration and TeamCity. Git flow The Badoo Team uses Git as a version control system. Our model ensures each task is developed and tested in a separate branch. The branch name consists of the ticket number in JIRA and a description of the problem. BFG-9000_All_developers_should
Introduction: At a Cloud Computing Meetup , Siddharth "Sid" Anand of Netflix, backed by a merry band of Netflixians, gave an interesting talk: Keeping Movies Running Amid Thunderstorms . While the talk gave a good overview of their move to the cloud, issues with capacity planning, thundering herds , latency problems, and simian armageddon , I found myself most taken with how they handle software deployment in the cloud . I've worked on half a dozen or more build and deployment systems, some small, some quite large, but never for a large organization like Netflix in the cloud. The cloud has this amazing capability that has never existed before that enables a novel approach to fault-tolerant software deployments: the ability to spin up huge numbers of instances to completely run a new release while running the old release at the same time . The process goes something like: A canary machine is launched first with the new software load running real traffic to sanity test the load in a p
4 0.63458788 138 high scalability-2007-10-30-Feedblendr Architecture - Using EC2 to Scale
Introduction: A man had a dream. His dream was to blend a bunch of RSS/Atom/RDF feeds into a single feed. The man is Beau Lebens of Feedville and like most dreamers he was a little short on coin. So he took refuge in the home of a cheap hosting provider and Beau realized his dream, creating FEEDblendr . But FEEDblendr chewed up so much CPU creating blended feeds that the cheap hosting provider ordered Beau to find another home. Where was Beau to go? He eventually found a new home in the virtual machine room of Amazon's EC2. This is the story of how Beau was finally able to create his one feeds safe within the cradle of affordable CPU cycles. Site: http://feedblendr.com/ The Platform EC2 (Fedora Core 6 Lite distro) S3 Apache PHP MySQL DynDNS (for round robin DNS) The Stats Beau is a developer with some sysadmin skills, not a web server admin, so a lot of learning was involved in creating FEEDblendr. FEEDblendr uses 2 EC2 instances. The same Amazon Instance (AMI) is
5 0.63444555 344 high scalability-2008-06-09-FaceStat's Rousing Tale of Scaling Woe and Wisdom Won
Introduction: Lukas Biewald shares a fascinating slam by slam recount of how his FaceStat (upload your picture and be judged by the masses) site was battered by a link on Yahoo's main page that caused an almost instantaneous 650,000 page view jump on their site. Yahoo spends considerable effort making sure its own properties can handle the truly massive flow from the main page. Turning the Great Eye of the Internet towards an unsuspecting newborn site must be quite the diaper ready experience. Theo Schlossnagle eerily prophesized about such events in The Implications of Punctuated Scalabilium for Website Architecture : massive, unexpected and sudden traffic spikes will become more common as a fickle internet seeks ever for new entertainments (my summary). Exactly FaceStat's situation. This is also one of our first exposures to an application written on Merb, a popular Ruby on Rails competitor. For those who think Ruby is the problem, their architecture now serves 100 times the original load
6 0.62743878 1335 high scalability-2012-10-08-How UltraDNS Handles Hundreds of Thousands of Zones and Tens of Millions of Records
7 0.62522531 1209 high scalability-2012-03-14-The Azure Outage: Time Is a SPOF, Leap Day Doubly So
8 0.61847627 659 high scalability-2009-07-20-A Scalability Lament
9 0.61790967 1517 high scalability-2013-09-16-The Hidden DNS Tax - Cascading Timeouts and Errors
10 0.61353046 244 high scalability-2008-02-11-Yahoo Live's Scaling Problems Prove: Release Early and Often - Just Don't Screw Up
12 0.59977484 1269 high scalability-2012-06-20-iDoneThis - Scaling an Email-based App from Scratch
13 0.5962624 1171 high scalability-2012-01-09-The Etsy Saga: From Silos to Happy to Billions of Pageviews a Month
14 0.59615642 429 high scalability-2008-10-25-Product: Puppet the Automated Administration System
15 0.58995092 1068 high scalability-2011-06-27-TripAdvisor Architecture - 40M Visitors, 200M Dynamic Page Views, 30TB Data
16 0.58826905 691 high scalability-2009-08-31-Squarespace Architecture - A Grid Handles Hundreds of Millions of Requests a Month
17 0.58633316 788 high scalability-2010-03-04-How MySpace Tested Their Live Site with 1 Million Concurrent Users
18 0.58218944 1077 high scalability-2011-07-11-ATMCash Exploits Virtualization for Security - Immutability and Reversion
19 0.58035618 1438 high scalability-2013-04-10-Check Yourself Before You Wreck Yourself - Avocado's 5 Early Stages of Architecture Evolution
20 0.5790596 378 high scalability-2008-09-03-Some Facebook Secrets to Better Operations
topicId topicWeight
[(1, 0.084), (2, 0.165), (61, 0.608)]
simIndex simValue blogId blogTitle
1 0.99485499 1303 high scalability-2012-08-13-Ask HighScalability: Facing scaling issues with news feeds on Redis. Any advice?
Introduction: We just released a social section to our iOS app several days ago and we are already facing scaling issues with the users' news feeds. We're basically using a Fan-out-on-write (push) model for the users' news feeds (posts of people and topics they follow) and we're using Redis for this (backend is Rails on Heroku). However, our current 60,000 news feeds is ballooning our Redis store to almost 1GB in a just a few days (it's growing way too fast for our budget). Currently we're storing the entire news feed for the user (post id, post text, author, icon url, etc) and we cap the entries to 300 per feed. I'm wondering if we need to just store the post IDs of each user feed in Redis and then store the rest of the post information somewhere else? Would love some feedback here. In this case, our iOS app would make an api call to our Rails app to retrieve a user's news feed. Rails app would retrieve news feed list (just post IDs) from Redis, and then Rails app would need to query to g
2 0.99183834 549 high scalability-2009-03-26-Performance - When do I start worrying?
Introduction: A common problem of the application designers is to predict when they need to start worrying about the Architectural/System improvements on their application. Do I need to add more resources? If yes, then how long before I am compelled to do so? The question is not only when but also what. Should I plan to implement a true caching layer on top of my application or do I need to shard my database. Do I need to move to a distributed search infrastructure and if yes when ! Essentially we try to find out the functionalities of the application that will become critical over time.
3 0.99077427 493 high scalability-2009-01-16-Just-In-Time Scalability: Agile Methods to Support Massive Growth (IMVU case study)
Introduction: Before We started with a small site, a mess of open source, and a small team that didn't know much about scaling. After We ended with a large site, a medium sized team, and an architecture that has scaled. We never stopped. We used a roadmap and a compass, made weekly changes in direction, regularly shipped code on Wednesday to handle the next weekend's capacity constraints, and shipped new features the whole time. These are excerpts from the IMVU PDF presentation of their architecture which can be viewed or downloaded here . IMVU is an online destination where adults and teens meet new people in 3D. IMVU won the 2008 Virtual Worlds Innovation Award and was also named a Rising Star in the 2008 Silicon Valley Technology Fast 50 program.
4 0.98493737 580 high scalability-2009-04-24-INFOSCALE 2009 in June in Hong Kong
Introduction: In case you are interested here's the info: INFOSCALE 2009: The 4th International ICST Conference on Scalable Information Systems. 10-12 June 2009, Hong Kong, China. In the last few years, we have seen the proliferation of the use of heterogeneous distributed systems, ranging from simple Networks of Workstations, to highly complex grid computing environments. Such computational paradigms have been preferred due to their reduced costs and inherent scalability, which pose many challenges to scalable systems and applications in terms of information access, storage and retrieval. Grid computing, P2P technology, data and knowledge bases, distributed information retrieval technology and networking technology should all converge to address the scalability concern. Furthermore, with the advent of emerging computing architectures - e.g. SMTs, GPUs, Multicores. - the importance of designing techniques explicitly targeting these systems is becoming more and more important. INFOSCA
5 0.98130423 793 high scalability-2010-03-10-Saying Yes to NoSQL; Going Steady with Cassandra at Digg
Introduction: The last six months have been exciting for Digg's engineering team. We're working on a soup-to-nuts rewrite. Not only are we rewriting all our application code, but we're also rolling out a new client and server architecture. And if that doesn't sound like a big enough challenge, we're replacing most of our infrastructure components and moving away from LAMP. Perhaps our most significant infrastructure change is abandoning MySQL in favor of a NoSQL alternative. To someone like me who's been building systems almost exclusively on relational databases for almost 20 years, this feels like a bold move. What's Wrong with MySQL? Our primary motivation for moving away from MySQL is the increasing difficulty of building a high performance, write intensive, application on a data set that is growing quickly, with no end in sight. This growth has forced us into horizontal and vertical partitioning strategies that have eliminated most of the value of a relational database, while still incurr
6 0.98103094 226 high scalability-2008-01-28-DR-BC for web-DB servers
7 0.97713548 746 high scalability-2009-11-26-Kngine Snippet Search New Indexing Technology
8 0.97705257 208 high scalability-2008-01-11-FTP Sanity: Redundancy, archiving, consolidation.
9 0.97661352 1201 high scalability-2012-02-29-Strategy: Put Mobile Video Into Cold Storage After 30 Days
10 0.97374624 324 high scalability-2008-05-19-UK Based CDN
same-blog 11 0.97191387 173 high scalability-2007-12-05-Easier Production Releases
12 0.97100788 930 high scalability-2010-10-28-NoSQL Took Away the Relational Model and Gave Nothing Back
13 0.96582234 675 high scalability-2009-08-08-1dbase vs. many and cloud hosting vs. dedicated server(s)?
14 0.95709431 268 high scalability-2008-03-06-Announce: First Meeting of Boston Scalability User Group
15 0.95674276 347 high scalability-2008-07-07-Five Ways to Stop Framework Fixation from Crashing Your Scaling Strategy
16 0.91951323 322 high scalability-2008-05-19-Conference: Infoscale 2008 in Italy (June 4-6)
17 0.90686154 238 high scalability-2008-02-04-IPS-IDS for heavy content site
18 0.90053475 198 high scalability-2008-01-01-HOW CDN works
19 0.89890736 1287 high scalability-2012-07-20-Stuff The Internet Says On Scalability For July 20, 2012
20 0.89583242 749 high scalability-2009-12-15-The Common Principles Behind the NOSQL Alternatives