high_scalability high_scalability-2008 high_scalability-2008-369 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: G'day, I'm building an application to manage WordPress PHP code on many servers. Our application will push down code updates to each server, as well as performing backups and testing. I'm considering different methods of pushing updated code onto the individual servers. I'm considering something like Capistrano (I've no experience in Ruby though). I've also considered using subversion and then remotely calling svn commands via SSH. Are there any other tools specifically for this purpose? The servers will have persistent data (the WordPress databases) so I don't want to re-image them every update. Plus, they will each have a different set of plugins / themes, so building many images would be too complex. If there are any papers on code deployment, or other recommended reading, please point the links my way. Likewise, if anyone has any suggestions, or would like more details, just let me know. Cheers - Callum .
sentIndex sentText sentNum sentScore
1 G'day, I'm building an application to manage WordPress PHP code on many servers. [sent-1, score-0.514]
2 Our application will push down code updates to each server, as well as performing backups and testing. [sent-2, score-0.694]
3 I'm considering different methods of pushing updated code onto the individual servers. [sent-3, score-1.134]
4 I'm considering something like Capistrano (I've no experience in Ruby though). [sent-4, score-0.317]
5 I've also considered using subversion and then remotely calling svn commands via SSH. [sent-5, score-1.101]
6 Are there any other tools specifically for this purpose? [sent-6, score-0.195]
7 The servers will have persistent data (the WordPress databases) so I don't want to re-image them every update. [sent-7, score-0.124]
8 Plus, they will each have a different set of plugins / themes, so building many images would be too complex. [sent-8, score-0.619]
9 If there are any papers on code deployment, or other recommended reading, please point the links my way. [sent-9, score-0.742]
10 Likewise, if anyone has any suggestions, or would like more details, just let me know. [sent-10, score-0.229]
wordName wordTfidf (topN-words)
[('wordpress', 0.42), ('considering', 0.256), ('svn', 0.215), ('subversion', 0.199), ('themes', 0.192), ('remotely', 0.192), ('capistrano', 0.19), ('code', 0.185), ('plugins', 0.185), ('recommended', 0.172), ('calling', 0.172), ('suggestions', 0.158), ('backups', 0.137), ('papers', 0.136), ('commands', 0.136), ('methods', 0.135), ('onto', 0.133), ('pushing', 0.126), ('specifically', 0.125), ('updated', 0.125), ('persistent', 0.124), ('purpose', 0.121), ('plus', 0.118), ('considered', 0.114), ('links', 0.113), ('performing', 0.112), ('ruby', 0.112), ('images', 0.109), ('building', 0.107), ('updates', 0.098), ('anyone', 0.097), ('reading', 0.096), ('individual', 0.095), ('deployment', 0.093), ('php', 0.091), ('push', 0.088), ('please', 0.081), ('details', 0.081), ('though', 0.08), ('different', 0.079), ('manage', 0.079), ('application', 0.074), ('via', 0.073), ('tools', 0.07), ('would', 0.07), ('many', 0.069), ('databases', 0.063), ('let', 0.062), ('experience', 0.061), ('point', 0.055)]
simIndex simValue blogId blogTitle
same-blog 1 1.0 369 high scalability-2008-08-18-Code deployment tools
Introduction: G'day, I'm building an application to manage WordPress PHP code on many servers. Our application will push down code updates to each server, as well as performing backups and testing. I'm considering different methods of pushing updated code onto the individual servers. I'm considering something like Capistrano (I've no experience in Ruby though). I've also considered using subversion and then remotely calling svn commands via SSH. Are there any other tools specifically for this purpose? The servers will have persistent data (the WordPress databases) so I don't want to re-image them every update. Plus, they will each have a different set of plugins / themes, so building many images would be too complex. If there are any papers on code deployment, or other recommended reading, please point the links my way. Likewise, if anyone has any suggestions, or would like more details, just let me know. Cheers - Callum .
2 0.17856698 255 high scalability-2008-02-21-Product: Capistrano - Automate Remote Tasks Via SSH
Introduction: Update: Deployment with Capistrano by Charles Max Wood. Nice simple step-by-step for using Capistrano for deployment. From their website: Simply put, Capistrano is a tool for automating tasks on one or more remote servers. It executes commands in parallel on all targeted machines, and provides a mechanism for rolling back changes across multiple machines. It is ideal for anyone doing any kind of system administration, either professionally or incidentally. * Great for automating tasks via SSH on remote servers, like software installation, application deployment, configuration management, ad hoc server monitoring, and more. * Ideal for system administrators, whether professional or incidental. * Easy to customize. Its configuration files use the Ruby programming language syntax, but you don't need to know Ruby to do most things with Capistrano. * Easy to extend. Capistrano is written in the Ruby programming language, and may be extended easily by writing additional Ruby mod
3 0.1035277 417 high scalability-2008-10-15-Outside.in Scales Up with Engine Yard and moving from PHP to Ruby on Rails
Introduction: This article explains how Outside.in , the local social network and aggregator, scaled up its service and moved from PHP to Ruby on Rails (they moved maybe because Ruby code seemed to be more maintanable that PHP code?). The whole article is here on EngineYard blog .
4 0.1002984 461 high scalability-2008-12-05-Sprinkle - Provisioning Tool to Build Remote Servers
Introduction: At 37 Signals Joshua Sierles describes how 37 Signals uses Sprinkle to configure their servers within EC2. Sprinkle defines a domain specific meta-language for describing and processing the installation of software . You can find an interesting discussion of Sprinkle's creation story by the creator himself, Marcus Crafter, in Sprinkle Some Powder! . Marcus divides provisioning tools into two categories: Task Based - the tool issues a list of commands to run on the remote system, either remotely via a network connection or smart client. Policy/state Based - the tool determines what needs to be run on the remote system by examining its current and final state. Sprinkle combines both models together in a chocolate-in-my-peanut-butter approach using normal Ruby code as the DSL (domain specific language) to declaratively describe remote system configurations. 37 Signals likes the use of Ruby as the DSL because it makes learning a separate syntax unnecessary. I've successfu
5 0.092558578 261 high scalability-2008-02-25-Make Your Site Run 10 Times Faster
Introduction: This is what Mike Peters says he can do : make your site run 10 times faster. His test bed is "half a dozen servers parsing 200,000 pages per hour over 40 IP addresses, 24 hours a day." Before optimization CPU spiked to 90% with 50 concurrent connections. After optimization each machine "was effectively handling 500 concurrent connections per second with CPU at 8% and no degradation in performance." Mike identifies six major bottlenecks: Database write access (read is cheaper) Database read access PHP, ASP, JSP and any other server side scripting Client side JavaScript Multiple/Fat Images, scripts or css files from different domains on your page Slow keep-alive client connections, clogging your available sockets Mike's solutions: Switch all database writes to offline processing Minimize number of database read access to the bare minimum. No more than two queries per page. Denormalize your database and Optimize MySQL tables Implement MemCached and cha
6 0.084953204 722 high scalability-2009-10-15-Hot Scalability Links for Oct 15 2009
7 0.083716772 887 high scalability-2010-08-24-Sponsored Post: deviantART, Okta, EzRez, Cloud Sigma, ManageEngine, Site24x7
8 0.082376301 405 high scalability-2008-10-07-Help a Scoble out. What should Robert ask in his scalability interview?
9 0.081977963 319 high scalability-2008-05-14-Scaling an image upload service
10 0.081949115 1063 high scalability-2011-06-17-Stuff The Internet Says On Scalability For June 17, 2011
11 0.081494182 1008 high scalability-2011-03-22-Facebook's New Realtime Analytics System: HBase to Process 20 Billion Events Per Day
12 0.080968708 876 high scalability-2010-08-10-Sponsored Post: Okta, EzRez, VoltDB, Digg, Cloud Sigma, Applications Manager, Site24x7
13 0.080518104 866 high scalability-2010-07-27-Sponsored Post: Okta, EzRez, VoltDB, Digg, Cloud Sigma, Applications Manager, Site24x7
14 0.076652311 1068 high scalability-2011-06-27-TripAdvisor Architecture - 40M Visitors, 200M Dynamic Page Views, 30TB Data
15 0.076016694 313 high scalability-2008-05-02-Friends for Sale Architecture - A 300 Million Page View-Month Facebook RoR App
16 0.075779326 910 high scalability-2010-09-30-Facebook and Site Failures Caused by Complex, Weakly Interacting, Layered Systems
17 0.075507365 1268 high scalability-2012-06-20-Ask HighScalability: How do I organize millions of images?
18 0.074564219 86 high scalability-2007-09-09-Clustering Solution
19 0.07074865 714 high scalability-2009-10-02-HighScalability has Moved to Squarespace.com!
20 0.070387758 1240 high scalability-2012-05-07-Startups are Creating a New System of the World for IT
topicId topicWeight
[(0, 0.118), (1, 0.042), (2, 0.001), (3, -0.058), (4, 0.041), (5, -0.017), (6, 0.005), (7, -0.029), (8, 0.004), (9, 0.021), (10, -0.045), (11, 0.048), (12, -0.002), (13, -0.062), (14, -0.0), (15, -0.05), (16, -0.013), (17, 0.025), (18, -0.03), (19, -0.017), (20, -0.007), (21, -0.028), (22, -0.035), (23, 0.034), (24, -0.001), (25, -0.001), (26, 0.02), (27, -0.053), (28, -0.038), (29, -0.003), (30, -0.047), (31, -0.012), (32, -0.011), (33, 0.049), (34, 0.002), (35, 0.013), (36, 0.036), (37, -0.01), (38, -0.0), (39, 0.036), (40, -0.041), (41, -0.017), (42, -0.012), (43, -0.005), (44, 0.041), (45, -0.003), (46, 0.032), (47, 0.008), (48, 0.021), (49, 0.003)]
simIndex simValue blogId blogTitle
same-blog 1 0.95284969 369 high scalability-2008-08-18-Code deployment tools
Introduction: G'day, I'm building an application to manage WordPress PHP code on many servers. Our application will push down code updates to each server, as well as performing backups and testing. I'm considering different methods of pushing updated code onto the individual servers. I'm considering something like Capistrano (I've no experience in Ruby though). I've also considered using subversion and then remotely calling svn commands via SSH. Are there any other tools specifically for this purpose? The servers will have persistent data (the WordPress databases) so I don't want to re-image them every update. Plus, they will each have a different set of plugins / themes, so building many images would be too complex. If there are any papers on code deployment, or other recommended reading, please point the links my way. Likewise, if anyone has any suggestions, or would like more details, just let me know. Cheers - Callum .
2 0.80256653 255 high scalability-2008-02-21-Product: Capistrano - Automate Remote Tasks Via SSH
Introduction: Update: Deployment with Capistrano by Charles Max Wood. Nice simple step-by-step for using Capistrano for deployment. From their website: Simply put, Capistrano is a tool for automating tasks on one or more remote servers. It executes commands in parallel on all targeted machines, and provides a mechanism for rolling back changes across multiple machines. It is ideal for anyone doing any kind of system administration, either professionally or incidentally. * Great for automating tasks via SSH on remote servers, like software installation, application deployment, configuration management, ad hoc server monitoring, and more. * Ideal for system administrators, whether professional or incidental. * Easy to customize. Its configuration files use the Ruby programming language syntax, but you don't need to know Ruby to do most things with Capistrano. * Easy to extend. Capistrano is written in the Ruby programming language, and may be extended easily by writing additional Ruby mod
3 0.74434352 218 high scalability-2008-01-17-Moving old to new. Do not be afraid of the re-write -- but take some help
Introduction: Recently I had to help users on one of my opensource project ISPMan. http://ispman.net This project started in 2001 as I was too unwilling to take care of the DNS and VitualHosting stuff as it was a side-thing to the company I worked for (so i wrote a software that took care of all these little details) Summary: A large project that needs a rewrite can be done in a matter of day. I will not give you a full case study about a project that went through a re-write but a case study about how easy it is to re-write something. Details: My boss was cool enough to let me open-source the project and obviously, I got a lot of cool-cred out of it. Later on I also did some support and implementation and earned quiet some money with it. Eventually I had to let the project go out of my hand to the community as I only did it to facilitate a job that wasnt williing to do. (Setup DNS zones of multiple servers, find out which host should host the website and put VirtualHost
Introduction: This is guest post by Michael DeHaan (@laserllama), a software developer and architect, on Ansible , a simple deployment, model-driven configuration management, and command execution framework. I owe High Scalability a great deal of credit for the idea behind my latest software project. I was reading about how an older tool I helped create, Func, was used at Tumblr , and it kicked some ideas into gear. This article is about what happened from that idea. My observation, which the article reinforced, was that many shops end up using a configuration management tool (Puppet, Chef, cfengine), a separate deployment tool (Capistrano, Fabric) and yet another separate ad-hoc task execution tool (Func, pssh, etc) because one class of tool historically hasn't been good at all three jobs. My other observation (not from the article) was that the whole "infrastructure as code" movement, while revolutionary, and definitely great for many, was probably secretly grating on a good number of
5 0.70515233 208 high scalability-2008-01-11-FTP Sanity: Redundancy, archiving, consolidation.
Introduction: Easy FTP redundancy and consolidation with the Open Source project Generic-FTP. Works with probably any Linux FTP Server (ProFTPD only one tested). Get rid of some single points of failure. A very easy to set up solution using scripts written in PHP. Tested thoroughly in a production environment.
6 0.69842356 993 high scalability-2011-02-22-Is Node.js Becoming a Part of the Stack? SimpleGeo Says Yes.
7 0.68969524 385 high scalability-2008-09-16-Product: Func - Fedora Unified Network Controller
8 0.67893851 307 high scalability-2008-04-21-Using Google AppEngine for a Little Micro-Scalability
9 0.66913474 366 high scalability-2008-08-17-Many updates against MySQL
10 0.66284788 295 high scalability-2008-04-02-Product: Supervisor - Monitor and Control Your Processes
11 0.6549325 219 high scalability-2008-01-21-Product: Hyperic
12 0.65255225 461 high scalability-2008-12-05-Sprinkle - Provisioning Tool to Build Remote Servers
13 0.64660013 118 high scalability-2007-10-09-High Load on production Webservers after Sourcecode sync
14 0.644862 228 high scalability-2008-01-28-Product: ISPMan Centralized ISP Management System
15 0.63791907 1580 high scalability-2014-01-15-Vedis - An Embedded Implementation of Redis Supporting Terabyte Sized Databases
16 0.63513076 1469 high scalability-2013-06-03-GOV.UK - Not Your Father's Stack
17 0.63075078 1531 high scalability-2013-10-13-AIDA: Badoo’s journey into Continuous Integration
18 0.62913525 1561 high scalability-2013-12-09-Site Moves from PHP to Facebook's HipHop, Now Pages Load in .6 Seconds Instead of Five
19 0.62343293 1288 high scalability-2012-07-23-Ask HighScalability: How Do I Build My MegaUpload + Itunes + YouTube Startup?
20 0.62301719 435 high scalability-2008-10-30-The case for functional decomposition
topicId topicWeight
[(1, 0.094), (2, 0.335), (10, 0.072), (30, 0.077), (39, 0.189), (61, 0.053), (79, 0.06)]
simIndex simValue blogId blogTitle
same-blog 1 0.93994182 369 high scalability-2008-08-18-Code deployment tools
Introduction: G'day, I'm building an application to manage WordPress PHP code on many servers. Our application will push down code updates to each server, as well as performing backups and testing. I'm considering different methods of pushing updated code onto the individual servers. I'm considering something like Capistrano (I've no experience in Ruby though). I've also considered using subversion and then remotely calling svn commands via SSH. Are there any other tools specifically for this purpose? The servers will have persistent data (the WordPress databases) so I don't want to re-image them every update. Plus, they will each have a different set of plugins / themes, so building many images would be too complex. If there are any papers on code deployment, or other recommended reading, please point the links my way. Likewise, if anyone has any suggestions, or would like more details, just let me know. Cheers - Callum .
2 0.91188264 1498 high scalability-2013-08-07-RAFT - In Search of an Understandable Consensus Algorithm
Introduction: If like many humans you've found even Paxos Made Simple a bit difficult to understand, you might enjoy RAFT as described in In Search of an Understandable Consensus Algorithm by Stanford's Diego Ongaro and John Ousterhout. The video presentation of the paper is given by John Ousterhout . Both the paper and the video are delightfully accessible. mcherm has a good summary of the paper: A consensus algorithm is: a cluster of servers should record a series of records ("log entries") in response to requests from clients of the cluster. (It may also take action based on those entries.) It does so in a way that guarantees that the responses seen by clients of the cluster will be consistent EVEN in the face of servers crashing in unpredictable ways (but not loosing data that was synched to disk), and networks introducing unpredictable delays or communication blockages. Here's what Raft does. First, it elects a leader, then the leader records the master version of the log, t
3 0.89333695 571 high scalability-2009-04-15-Using HTTP cache headers effectively
Introduction: Hi, Some time ago , martin fowler bloged about how HTTP cache headers can be very effectively used in web site design. http://www.martinfowler.com/bliki/SegmentationByFreshness.html How actively HTTP cache headers are considered in web site design? I think it is a great tool to reduce lot of load on server and should be considered before designing any complex caching strategy. Thoughts? Thanks, Unmesh
4 0.88186777 901 high scalability-2010-09-16-How Can the Large Hadron Collider Withstand One Petabyte of Data a Second?
Introduction: Why is there something rather than nothing? That's the kind of question the Large Hadron Collider in CERN is hopefully poised to answer. And what is the output of this beautiful 17-mile long, 6 billion dollar wabi-sabish proton smashing machine? Data. Great heaping torrents of Grand Canyon sized data. 15 million gigabytes every year. That's 1000 times the information printed in books every year. It's so much data 10,000 scientists will use a grid of 80,000+ computers , in 300 computer centers , in 50 different countries just to help make sense of it all. How will all this data be collected, transported, stored, and analyzed? It turns out, using what amounts to sort of Internet of Particles instead of an Internet of Things. Two good articles have recently shed some electro-magnetic energy in the human visible spectrum on the IT aspects of the collider: LHC computing grid pushes petabytes of data, beats expectations by John Timmer on Ars Technica and an overview of the Br
5 0.87713462 1373 high scalability-2012-12-17-11 Uses For the Humble Presents Queue, er, Message Queue
Introduction: It's a little known fact that Santa Clause was an early queue innovator. Faced with the problem of delivering a planet full of presents in one night, Santa, in his hacker's workshop, created a Present Distribution System using thousands of region based priority present queues for continuous delivery by the Rudolphs. Rudolphs? You didn't think there was only one Rudolph did you? Presents are delivered in parallel by a cluster of sleighs, each with redundant reindeer in a master-master configuration. Each Rudolph is a cluster leader and they coordinate work using an early and more magical version of the ZooKeeper protocol. Programmers have followed Santa's lead and you can find a message queue in nearly every major architecture profile on HighScalability . Historically they may have been introduced after a first generation architecture needed to scale up from their two tier system into something a little more capable (asynchronicity, work dispatch, load buffering, database offloadin
6 0.87445509 1145 high scalability-2011-11-18-Stuff The Internet Says On Scalability For November 18, 2011
7 0.87233967 464 high scalability-2008-12-13-Strategy: Facebook Tweaks to Handle 6 Time as Many Memcached Requests
8 0.870848 406 high scalability-2008-10-08-Strategy: Flickr - Do the Essential Work Up-front and Queue the Rest
9 0.86951399 1346 high scalability-2012-10-24-Saving Cash Using Less Cache - 90% Savings in the Caching Tier
10 0.86935496 721 high scalability-2009-10-13-Why are Facebook, Digg, and Twitter so hard to scale?
11 0.86875099 1006 high scalability-2011-03-17-Are long VM instance spin-up times in the cloud costing you money?
12 0.8685562 1387 high scalability-2013-01-15-More Numbers Every Awesome Programmer Must Know
14 0.86532694 1456 high scalability-2013-05-13-The Secret to 10 Million Concurrent Connections -The Kernel is the Problem, Not the Solution
15 0.86507332 1413 high scalability-2013-02-27-42 Monster Problems that Attack as Loads Increase
16 0.86368883 221 high scalability-2008-01-24-Mailinator Architecture
17 0.86292809 1199 high scalability-2012-02-27-Zen and the Art of Scaling - A Koan and Epigram Approach
18 0.86256981 1126 high scalability-2011-09-27-Use Instance Caches to Save Money: Latency == $$$
20 0.86221957 1425 high scalability-2013-03-18-Beyond Threads and Callbacks - Application Architecture Pros and Cons