high_scalability high_scalability-2008 high_scalability-2008-429 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Update: Digg on their choice and use of Puppet . They chose puppet over cfengine, and bcfg2 because they liked Puppet's resource abstraction layer (RAL), the ability to implement configuration management incrementally, support for bundles, and the overall design philosophy. Puppet implements a declarative (what not how) configuration language for automating common administration tasks. It's the system every large site writes for themselves and it's already made for you! Ilike was able to "easily" scale from 0 to hundreds of servers using Puppet. I can't believe I've never seen this before. It looks really cool. What is Puppet and how can it help you scale your website operations? From the Puppet website: Puppet has been developed to help the sysadmin community move to building and sharing mature tools that avoid the duplication of everyone solving the same problem. It does so in two ways: * It provides a powerful framework to simplify the majority of the technical tasks t
sentIndex sentText sentNum sentScore
1 They chose puppet over cfengine, and bcfg2 because they liked Puppet's resource abstraction layer (RAL), the ability to implement configuration management incrementally, support for bundles, and the overall design philosophy. [sent-2, score-0.823]
2 Puppet implements a declarative (what not how) configuration language for automating common administration tasks. [sent-3, score-0.354]
3 What is Puppet and how can it help you scale your website operations? [sent-8, score-0.059]
4 From the Puppet website: Puppet has been developed to help the sysadmin community move to building and sharing mature tools that avoid the duplication of everyone solving the same problem. [sent-9, score-0.626]
5 It does so in two ways: * It provides a powerful framework to simplify the majority of the technical tasks that sysadmins need to perform * The sysadmin work is written as code in Puppet's custom language which is shareable just like any other code. [sent-10, score-0.757]
6 This means that your work as a sysadmin can get done much faster, because you can have Puppet handle most or all of the details, and you can download code from other sysadmins to help you get done even faster. [sent-11, score-0.481]
7 The majority of Puppet implementations use at least one or two modules developed by someone else, and there are already tens of recipes available in Puppet's CookBook. [sent-12, score-0.281]
8 HJK Solutions' Adam Jacob says it does : Puppet enables us to get a huge jump-start on building automated, scaleable, easy to manage infrastructures for our clients. [sent-15, score-0.13]
9 Automate as much of the routine systems administration tasks as possible. [sent-17, score-0.25]
10 Get 10 minute unattended build times from bare metal, most of which is data transfer. [sent-19, score-0.146]
11 Bootstrap our clients production environments while building their development environment. [sent-23, score-0.146]
12 Because we are expressing the infrastructure at a higher level, when it comes time to deploy your production systems, it’s really a non-event. [sent-25, score-0.143]
13 We just roll out the Puppet Master and an Operating System auto-install environment, and it’s finished. [sent-26, score-0.05]
14 We work with several different shops using Ruby on Rails, all of whom have very similar infrastructure needs. [sent-29, score-0.24]
15 By using Puppet in all of them, when we solve a problem for one client, we’ve effectively solved it for the others. [sent-30, score-0.088]
16 I love being able to tell a client that we solved a problem for them, and all it’s going to cost is the time it takes for us to add the recipe. [sent-31, score-0.143]
17 Puppet, today, is a tool that is good enough to handle the vast majority of issues encountered in building scalable infrastructures. [sent-32, score-0.288]
18 Even the places where it falls short are almost always just a matter of it being less elegant than it could be, and the entire community is working on making those parts better. [sent-33, score-0.188]
19 0 by John Willis Why are people paying 3 to 5 million for configuration management software? [sent-40, score-0.077]
wordName wordTfidf (topN-words)
[('puppet', 0.694), ('sysadmin', 0.22), ('ilike', 0.178), ('jacob', 0.17), ('sysadmins', 0.15), ('majority', 0.147), ('adam', 0.124), ('administration', 0.1), ('breakfrom', 0.095), ('saysit', 0.095), ('john', 0.09), ('unattended', 0.089), ('solved', 0.088), ('expressing', 0.085), ('cfengine', 0.085), ('jesse', 0.082), ('routine', 0.079), ('configuration', 0.077), ('building', 0.075), ('duplication', 0.073), ('developed', 0.071), ('tasks', 0.071), ('clients', 0.071), ('shops', 0.071), ('encountered', 0.066), ('community', 0.066), ('bootstrap', 0.064), ('elegant', 0.064), ('recipes', 0.063), ('mature', 0.062), ('metal', 0.061), ('language', 0.06), ('help', 0.059), ('declarative', 0.059), ('similar', 0.059), ('sauce', 0.058), ('falls', 0.058), ('infrastructure', 0.058), ('automating', 0.058), ('simplify', 0.057), ('bare', 0.057), ('client', 0.055), ('stress', 0.055), ('infrastructures', 0.055), ('incrementally', 0.055), ('automate', 0.053), ('work', 0.052), ('liked', 0.052), ('roll', 0.05), ('digg', 0.05)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999994 429 high scalability-2008-10-25-Product: Puppet the Automated Administration System
Introduction: Update: Digg on their choice and use of Puppet . They chose puppet over cfengine, and bcfg2 because they liked Puppet's resource abstraction layer (RAL), the ability to implement configuration management incrementally, support for bundles, and the overall design philosophy. Puppet implements a declarative (what not how) configuration language for automating common administration tasks. It's the system every large site writes for themselves and it's already made for you! Ilike was able to "easily" scale from 0 to hundreds of servers using Puppet. I can't believe I've never seen this before. It looks really cool. What is Puppet and how can it help you scale your website operations? From the Puppet website: Puppet has been developed to help the sysadmin community move to building and sharing mature tools that avoid the duplication of everyone solving the same problem. It does so in two ways: * It provides a powerful framework to simplify the majority of the technical tasks t
2 0.4514375 1408 high scalability-2013-02-19-Puppet monitoring: how to monitor the success or failure of Puppet runs
Introduction: This is a guest post by LogicMonitor's Director of Tech Ops, Jesse Aukeman , about the different ways they're monitoring the success or failure of Puppet runs. If you are like us, you are running some type of linux configuration management tool. The value of centralized configuration and deployment is well known and hard to overstate. Puppet is our tool of choice. It is powerful and works well for us, except when things don't go as planned. Failures of puppet can be innocuous and cosmetic, or they can cause production issues, for example when crucial updates do not get properly propagated. Why? In the most innocuous cases, the puppet agent craps out (we run puppet agent via cron). As nice as puppet is, we still need to goose it from time to time to get past some sort of network or host resource issue. A more dangerous case is when an administrator temporarily disables puppet runs on a host in order to perform some test or administrative task and then forgets to reenab
3 0.26893693 1058 high scalability-2011-06-13-Automation on AWS with Ruby and Puppet
Introduction: This is a guest post by Frédéric Faure (architect at Ysance ), you can follow him on twitter . Urbandive is an immersive view service launched by the French YellowPages which allows you to travel in cities in France thanks to a 360° view. Urbandive focuses on providing high definition pictures and accurate professional and social content. One of the biggest jobs was to enable a fast scalable architecture, because it was really difficult to forecast the traffic load at production time. Traffic load may be influenced if the service receives attention from users as a result of advertising. Below you will find a summary of the goals we achieve by using a Ruby scheduler built on top of Puppet on AWS to create a complete infrastructure. Workflow & XTR-Lucid Our scalability combo is : a home-made Ruby scheduler ( XTR-Lucid ) to deal with AWS APIs + the Puppet Master to install services and configure EC2 instances and keep them up-to-date during all the product
4 0.17143744 1334 high scalability-2012-10-04-Stuff The Internet Says On Scalability For October 5, 2012
Introduction: It's HighScalability Time: 30 Million : Lady Gaga Twitter followers; 1 Billion : active Facebook users; Quotable Quotes: @mappingbabel : Oracle exec says "we're not competing with Amazon for Netflix, we're competing with Amazon for Boeing," @CompSciFact : "Most software looks more like a whirlpool than a pipeline." #gotoaar Peter Hunt : My experience with SSDs and Zookeeper has been discouraging. SSDs have some really terrible corner cases for latency. I've seen them take 40+ seconds (that's not a mistake - seconds) for fsync to complete. @kief : GCS uses AWS terminology for regions and zones, are these becoming industry standard? #velocityconf @ibogost : When someone says "Big Data," I always check to see if I still have my wallet. @samnewman : #newrelic went from 10 servers with 0 customers, to 12 servers supporting 5TB, 58 billion metrics a day - @bjorn_fb at #gotoaar @beezly : G
5 0.1705216 881 high scalability-2010-08-16-Scaling an AWS infrastructure - Tools and Patterns
Introduction: This is a guest post by Frédéric Faure (architect at Ysance ), you can follow him on twitter . How do you scale an AWS (Amazon Web Services) infrastructure? This article will give you a detailed reply in two parts: the tools you can use to make the most of Amazon’s dynamic approach, and the architectural model you should adopt for a scalable infrastructure. I base my report on my experience gained in several AWS production projects in casual gaming (Facebook), e-commerce infrastructures and within the mainstream GIS (Geographic Information System). It’s true that my experience in gaming ( IsCool, The Game ) is currently the most representative in terms of scalability, due to the number of users (over 800 thousand DAU – daily active users – at peak usage and over 20 million page views every day), however my experiences in e-commerce and GIS (currently underway) provide a different view of scalability, taking into account the various problems of availability and da
6 0.15085155 461 high scalability-2008-12-05-Sprinkle - Provisioning Tool to Build Remote Servers
7 0.14490283 802 high scalability-2010-04-01-Hot Scalability Links for April 1, 2010
9 0.11471264 1240 high scalability-2012-05-07-Startups are Creating a New System of the World for IT
10 0.10890757 1522 high scalability-2013-09-25-Great Open Source Solution for Boring HA and Scalability Problems
11 0.10847277 1230 high scalability-2012-04-18-Ansible - A Simple Model-Driven Configuration Management and Command Execution Framework
14 0.10177793 1469 high scalability-2013-06-03-GOV.UK - Not Your Father's Stack
15 0.096891597 385 high scalability-2008-09-16-Product: Func - Fedora Unified Network Controller
topicId topicWeight
[(0, 0.158), (1, 0.031), (2, -0.018), (3, 0.004), (4, 0.04), (5, -0.044), (6, 0.004), (7, -0.012), (8, -0.015), (9, -0.015), (10, -0.011), (11, 0.052), (12, 0.056), (13, -0.054), (14, 0.04), (15, -0.049), (16, 0.021), (17, 0.017), (18, -0.014), (19, 0.026), (20, 0.003), (21, -0.017), (22, 0.001), (23, 0.006), (24, -0.032), (25, 0.029), (26, -0.012), (27, -0.012), (28, -0.086), (29, -0.013), (30, -0.016), (31, 0.023), (32, -0.002), (33, 0.036), (34, -0.01), (35, -0.041), (36, -0.041), (37, -0.014), (38, -0.012), (39, 0.086), (40, -0.067), (41, -0.028), (42, -0.044), (43, -0.004), (44, -0.019), (45, -0.044), (46, -0.08), (47, 0.042), (48, 0.015), (49, 0.043)]
simIndex simValue blogId blogTitle
same-blog 1 0.94881564 429 high scalability-2008-10-25-Product: Puppet the Automated Administration System
Introduction: Update: Digg on their choice and use of Puppet . They chose puppet over cfengine, and bcfg2 because they liked Puppet's resource abstraction layer (RAL), the ability to implement configuration management incrementally, support for bundles, and the overall design philosophy. Puppet implements a declarative (what not how) configuration language for automating common administration tasks. It's the system every large site writes for themselves and it's already made for you! Ilike was able to "easily" scale from 0 to hundreds of servers using Puppet. I can't believe I've never seen this before. It looks really cool. What is Puppet and how can it help you scale your website operations? From the Puppet website: Puppet has been developed to help the sysadmin community move to building and sharing mature tools that avoid the duplication of everyone solving the same problem. It does so in two ways: * It provides a powerful framework to simplify the majority of the technical tasks t
2 0.806481 461 high scalability-2008-12-05-Sprinkle - Provisioning Tool to Build Remote Servers
Introduction: At 37 Signals Joshua Sierles describes how 37 Signals uses Sprinkle to configure their servers within EC2. Sprinkle defines a domain specific meta-language for describing and processing the installation of software . You can find an interesting discussion of Sprinkle's creation story by the creator himself, Marcus Crafter, in Sprinkle Some Powder! . Marcus divides provisioning tools into two categories: Task Based - the tool issues a list of commands to run on the remote system, either remotely via a network connection or smart client. Policy/state Based - the tool determines what needs to be run on the remote system by examining its current and final state. Sprinkle combines both models together in a chocolate-in-my-peanut-butter approach using normal Ruby code as the DSL (domain specific language) to declaratively describe remote system configurations. 37 Signals likes the use of Ruby as the DSL because it makes learning a separate syntax unnecessary. I've successfu
Introduction: This is guest post by Michael DeHaan (@laserllama), a software developer and architect, on Ansible , a simple deployment, model-driven configuration management, and command execution framework. I owe High Scalability a great deal of credit for the idea behind my latest software project. I was reading about how an older tool I helped create, Func, was used at Tumblr , and it kicked some ideas into gear. This article is about what happened from that idea. My observation, which the article reinforced, was that many shops end up using a configuration management tool (Puppet, Chef, cfengine), a separate deployment tool (Capistrano, Fabric) and yet another separate ad-hoc task execution tool (Func, pssh, etc) because one class of tool historically hasn't been good at all three jobs. My other observation (not from the article) was that the whole "infrastructure as code" movement, while revolutionary, and definitely great for many, was probably secretly grating on a good number of
4 0.7788837 1408 high scalability-2013-02-19-Puppet monitoring: how to monitor the success or failure of Puppet runs
Introduction: This is a guest post by LogicMonitor's Director of Tech Ops, Jesse Aukeman , about the different ways they're monitoring the success or failure of Puppet runs. If you are like us, you are running some type of linux configuration management tool. The value of centralized configuration and deployment is well known and hard to overstate. Puppet is our tool of choice. It is powerful and works well for us, except when things don't go as planned. Failures of puppet can be innocuous and cosmetic, or they can cause production issues, for example when crucial updates do not get properly propagated. Why? In the most innocuous cases, the puppet agent craps out (we run puppet agent via cron). As nice as puppet is, we still need to goose it from time to time to get past some sort of network or host resource issue. A more dangerous case is when an administrator temporarily disables puppet runs on a host in order to perform some test or administrative task and then forgets to reenab
5 0.77285641 385 high scalability-2008-09-16-Product: Func - Fedora Unified Network Controller
Introduction: Func is used to manage a large network using bash or Python scripts. It targets easy and simple remote scripting and one-off tasks over SSH by creating a secure (SSL certifications) XMLRPC API for communication. Any kind of application can be written on top of it. Other configuration management tools specialize in mass configuration. They say here's what the machine should look like and keep it that way. Func allows you to program your cluster. If you've ever tried to securely remote script a gang of machines using SSH keys you know what a total nightmare that can be. Some example commands: Using the command line: func "*.example.org" call yumcmd update Using the Pthon API: import func.overlord.client as fc client = fc.Client("*.example.org;*.example.com") client.yumcmd.update() client.service.start("acme-server") print client.hardware.info() Func may certainly overlap in functionality with other tools like Puppet and cfengine, but as programmers we always need more than one
6 0.77180171 807 high scalability-2010-04-09-Vagrant - Build and Deploy Virtualized Development Environments Using Ruby
7 0.7470113 1531 high scalability-2013-10-13-AIDA: Badoo’s journey into Continuous Integration
8 0.72350472 255 high scalability-2008-02-21-Product: Capistrano - Automate Remote Tasks Via SSH
9 0.70285219 245 high scalability-2008-02-12-Product: rPath - Creating and Managing Virtual Appliances
10 0.69137961 1469 high scalability-2013-06-03-GOV.UK - Not Your Father's Stack
11 0.68969917 484 high scalability-2009-01-05-Lessons Learned at 208K: Towards Debugging Millions of Cores
12 0.68299294 1058 high scalability-2011-06-13-Automation on AWS with Ruby and Puppet
13 0.67723233 433 high scalability-2008-10-29-CTL - Distributed Control Dispatching Framework
14 0.67444545 219 high scalability-2008-01-21-Product: Hyperic
15 0.65747112 272 high scalability-2008-03-08-Product: FAI - Fully Automatic Installation
16 0.65237105 1422 high scalability-2013-03-12-If Your System was a Symphony it Might Sound Like This...
17 0.6379931 228 high scalability-2008-01-28-Product: ISPMan Centralized ISP Management System
18 0.63641733 1379 high scalability-2012-12-31-Designing for Resiliency will be so 2013
19 0.63473684 263 high scalability-2008-02-27-Product: System Imager - Automate Deployment and Installs
20 0.63190275 1628 high scalability-2014-04-08-Microservices - Not a free lunch!
topicId topicWeight
[(1, 0.215), (2, 0.163), (10, 0.038), (15, 0.011), (30, 0.023), (40, 0.013), (52, 0.161), (56, 0.015), (61, 0.102), (66, 0.015), (79, 0.048), (85, 0.074), (94, 0.017)]
simIndex simValue blogId blogTitle
1 0.93669915 47 high scalability-2007-07-30-Product: Yslow to speed up your web pages
Introduction: Update : Speed up Apache - how I went from F to A in YSlow . Good example of using YSlow to speed up a website with solid code examples. Every layer in the multi-layer cake that is your website contributes to how long a page takes to display. YSlow , from Yahoo, is a cool tool for discovering how the ingredients of your site's top layer contribute to performance. YSlow analyzes web pages and tells you why they're slow based on the rules for high performance web sites. YSlow is a Firefox add-on integrated with the popular Firebug web development tool. YSlow gives you: Performance report card HTTP/HTML summary List of components in the page Tools including JSLint
same-blog 2 0.93037599 429 high scalability-2008-10-25-Product: Puppet the Automated Administration System
Introduction: Update: Digg on their choice and use of Puppet . They chose puppet over cfengine, and bcfg2 because they liked Puppet's resource abstraction layer (RAL), the ability to implement configuration management incrementally, support for bundles, and the overall design philosophy. Puppet implements a declarative (what not how) configuration language for automating common administration tasks. It's the system every large site writes for themselves and it's already made for you! Ilike was able to "easily" scale from 0 to hundreds of servers using Puppet. I can't believe I've never seen this before. It looks really cool. What is Puppet and how can it help you scale your website operations? From the Puppet website: Puppet has been developed to help the sysadmin community move to building and sharing mature tools that avoid the duplication of everyone solving the same problem. It does so in two ways: * It provides a powerful framework to simplify the majority of the technical tasks t
3 0.92313761 1072 high scalability-2011-07-01-TripAdvisor Strategy: No Architects, Engineers Work Across the Entire Stack
Introduction: If you are an insect , don't work at TripAdvisor, specialization is out. One of the most commented on strategies from the TripAdvisor architecture article is their rather opinionated take on the role of engineers in the organization. Typically engineers live in a box. They are specialized, they do database work and not much else, and they just do programming, not much else. TripAdvisor takes the road less traveled: Engineers work across entire stack - HTML, CSS, JS, Java, scripting. If you do not know something, you learn it. The only thing that gets in the way of delivering your project is you, as you are expected to work at all levels - design, code, test, monitoring, CSS, JS, Java, SQL, scripting. We do not have "architects." At TripAdvisor, if you design something, your code it, and if you code it you test it. Engineers who do not like to go outside their comfort zone, or who feel certain work is "beneath" them will simply get in the way. A radical take for an e
4 0.90988141 872 high scalability-2010-08-05-Pairing NoSQL and Relational Data Storage: MySQL with MongoDB
Introduction: I’ve largely steered clear of publicly commenting on the “NoSQL vs. Relational” conflict. Keeping in mind that this argument is more about currently available solutions and the features their developers have chosen to build in, I’d like to dig into this and provide a decidedly neutral viewpoint. In fact, by erring on the side of caution, I’ve inadvertently given myself plenty of time to consider the pros and cons of both data storage approaches, and although my mind was initially swaying toward the NoSQL camp, I can say with a fair amount of certainty, that I’ve found a good compromise. You can read the full store here .
Introduction: Scoble the Ubiquitous has a fascinating post on how Mogulus, a live video channel startup, uses S3/EC2 and doesn't own a single server. The trends that have been happening for a while now are going mainstream. To do great things you no longer need to start by creating a huge war chest. You can forage off the land, like any good mobile, light weight fighting unit. For a strategy hit he mentions the same needed change in perspective as Beau Lebens talked about when making FeedBlendr : One tip he gave us is that when using Amazon’s services you have to design your systems with the assumption that they will never be up and running. What he means by that is services are “volatile” and can go up and down without notice. So, he’s designed his systems to survive that. He told me that it meant his engineering teams had to be quite disciplined in designing their architecture.
6 0.85842979 1432 high scalability-2013-04-01-Khan Academy Checkbook Scaling to 6 Million Users a Month on GAE
7 0.8581605 466 high scalability-2008-12-16-Facebook is Hiring
8 0.85646623 1399 high scalability-2013-02-05-Ask HighScalability: Memcached and Relations
9 0.85514653 244 high scalability-2008-02-11-Yahoo Live's Scaling Problems Prove: Release Early and Often - Just Don't Screw Up
10 0.85510087 918 high scalability-2010-10-12-The CIO’s Problem: Cloud “Mess” or Cloud “Mash”
11 0.85308677 824 high scalability-2010-05-06-Going global on EC2
12 0.85251725 707 high scalability-2009-09-17-Hot Links for 2009-9-17
13 0.85225135 1390 high scalability-2013-01-21-Processing 100 Million Pixels a Day - Small Amounts of Contention Cause Big Problems at Scale
14 0.85132527 1082 high scalability-2011-07-18-New Relic Architecture - Collecting 20+ Billion Metrics a Day
15 0.8511892 578 high scalability-2009-04-23-Which Key value pair database to be used
16 0.85077447 603 high scalability-2009-05-19-Scaling Memcached: 500,000+ Operations-Second with a Single-Socket UltraSPARC T2
17 0.84979504 688 high scalability-2009-08-26-Hot Links for 2009-8-26
18 0.84877282 720 high scalability-2009-10-12-High Performance at Massive Scale – Lessons learned at Facebook
19 0.84859478 924 high scalability-2010-10-21-What is Network-based Application Virtualization and Why Do You Need It?
20 0.84697407 70 high scalability-2007-08-22-How many machines do you need to run your site?