high_scalability high_scalability-2013 high_scalability-2013-1492 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Someone reading my now ancient C++ coding standard recommendation for using doxygen to automatically generate documentation from source code, asked a great question: I've often considered using doxygen, I always ask myself - is this really useful? Would I use it if I were new to a project? Would programmers working on the project use it? I'll rephrase their question to more conveniently express a point I've thought a lot about: Why do companies put so little effort into automating their own development process to make development easier? It's like the hair stylist whose own hair looks like someone cut it using a late night infomercial vacuum cleaner attachment. Or it's like the interior decorator whose own house looks like a monk's cell. Software organizations rarely build software to make developing software easier. Why is that? Because there are three ways changes are made in an organization: Top Dog - Someone so high up in the org chart you would need
sentIndex sentText sentNum sentScore
1 Someone reading my now ancient C++ coding standard recommendation for using doxygen to automatically generate documentation from source code, asked a great question: I've often considered using doxygen, I always ask myself - is this really useful? [sent-1, score-0.474]
2 You probably won't get enough time to do a good job so you end up with one hack built on top of another hack because people can't assign value to all the infrastructure that often makes the critical difference between project success or failure. [sent-29, score-0.489]
3 On learning the tool they may want to use won't be improved the new group has a choice: improve the existing tool or make something new. [sent-33, score-0.396]
4 Even worse and more likely is that people in an organization don't even try anymore because they know any efforts to make the development process better won't work out in the end. [sent-38, score-0.528]
5 They say when bears are thriving in an area it means the ecosystem in that area is pretty healthy because bears are at the top of the food chain. [sent-42, score-0.537]
6 But automated documentation generation is not so much about the technology needed to make it happen, it's more about the culture needed to maintain it and make it useful. [sent-46, score-0.727]
7 Automated documentation assumes people wrote the documentation to begin with. [sent-48, score-0.781]
8 It indicates that someone thought about what they were doing enough that they could explain it coherently to someone else. [sent-51, score-0.6]
9 It means that when someone wants to know if a capability is available then they can do a search of the documentation and hopefully find what they are looking for. [sent-52, score-0.458]
10 You can make a similar case about many other development practices: unit tests, source code control, bug tracking, training, having a methodology, automated builds, automated regression tests, continuous deployment, monitoring, auto-provisioning, etc etc. [sent-56, score-0.58]
11 But if these practices are in place it indicates a pretty healthy development ecosystem in which people are probably pretty productive. [sent-58, score-0.636]
12 Another tool that doesn't get enough use is a company or development wide wiki (or equivalent). [sent-59, score-0.455]
13 And again the active use of a wiki indicates a healthy development ecosystem. [sent-61, score-0.448]
14 Let's take the automated documentation system as an example of how to apply the ideas. [sent-67, score-0.495]
15 Come up with courses and good quick online doc for telling people how to create the doc. [sent-78, score-0.501]
16 Enmesh the Automated Document Generation In Company Processes Make the doc searchable so people will find themselves turning it when they have a question Make it required. [sent-82, score-0.555]
17 When someone has a question you have to be able to refer them to the doc most of the time. [sent-85, score-0.616]
18 Tie it into the build system so every nightly and release build has a link to the doc for the build. [sent-88, score-0.585]
19 Evangelize When you go a class and it doesn't have doc ask the author to put doc in. [sent-89, score-0.894]
20 When the doc sux ask the author to improve it. [sent-90, score-0.519]
wordName wordTfidf (topN-words)
[('doc', 0.375), ('roi', 0.298), ('documentation', 0.271), ('someone', 0.187), ('automated', 0.166), ('bears', 0.151), ('copyright', 0.144), ('wiki', 0.141), ('wikis', 0.128), ('people', 0.126), ('doxygen', 0.118), ('monkey', 0.118), ('enough', 0.115), ('assumes', 0.113), ('indicates', 0.111), ('tool', 0.111), ('healthy', 0.108), ('make', 0.105), ('footer', 0.101), ('organization', 0.096), ('hair', 0.09), ('development', 0.088), ('ask', 0.085), ('project', 0.084), ('hack', 0.082), ('culture', 0.08), ('nightly', 0.08), ('spreads', 0.076), ('continual', 0.073), ('adopted', 0.072), ('add', 0.072), ('character', 0.071), ('practices', 0.069), ('group', 0.069), ('extra', 0.068), ('pretty', 0.067), ('build', 0.065), ('useful', 0.064), ('adopt', 0.064), ('wo', 0.063), ('corporate', 0.063), ('food', 0.06), ('author', 0.059), ('anymore', 0.059), ('apply', 0.058), ('changes', 0.057), ('ca', 0.056), ('code', 0.055), ('process', 0.054), ('question', 0.054)]
simIndex simValue blogId blogTitle
same-blog 1 1.0000001 1492 high scalability-2013-07-17-How do you create a 100th Monkey software development culture?
Introduction: Someone reading my now ancient C++ coding standard recommendation for using doxygen to automatically generate documentation from source code, asked a great question: I've often considered using doxygen, I always ask myself - is this really useful? Would I use it if I were new to a project? Would programmers working on the project use it? I'll rephrase their question to more conveniently express a point I've thought a lot about: Why do companies put so little effort into automating their own development process to make development easier? It's like the hair stylist whose own hair looks like someone cut it using a late night infomercial vacuum cleaner attachment. Or it's like the interior decorator whose own house looks like a monk's cell. Software organizations rarely build software to make developing software easier. Why is that? Because there are three ways changes are made in an organization: Top Dog - Someone so high up in the org chart you would need
2 0.14048895 840 high scalability-2010-06-10-The Four Meta Secrets of Scaling at Facebook
Introduction: Aditya Agarwal, Director of Engineering at Facebook, gave an excellent Scale at Facebook talk that covers their architecture, but the talk is really more about how to scale an organization by preserving the best parts of its culture. The key take home of the talk is: You can get the code right, you can get the products right, but you need to get the culture right first. If you don't get the culture right then your company won't scale. This leads into the four meta secrets of scaling at Facebook: Scaling takes Iteration Don't Over Design Choose the right tool for the job, but realize that your choice comes with overhead. Get the culture right. Move Fast - break things. Huge Impact - small teams. Be bold - innovate. Some Background Facebook is big : 400 million active users; users spend an average of 20 minutes a day; 5 billion pieces of content (status updates, comments, likes, photo uploads, video uploads, chat messages, inbox messages, group events, f
3 0.13981576 1240 high scalability-2012-05-07-Startups are Creating a New System of the World for IT
Introduction: It remains that, from the same principles, I now demonstrate the frame of the System of the World. -- Isaac Newton The practice of IT reminds me a lot of the practice of science before Isaac Newton. Aristotelianism was dead, but there was nothing to replace it. Then Newton came along, created a scientific revolution with his System of the World . And everything changed. That was New System of the World number one. New System of the World number two was written about by the incomparable Neal Stephenson in his incredible Baroque Cycle series. It explores the singular creation of a new way of organizing society grounded in new modes of thought in business, religion, politics, and science. Our modern world emerged Enlightened as it could from this roiling cauldron of forces. In IT we may have had a Leonardo da Vinci or even a Galileo, but we’ve never had our Newton. Maybe we don't need a towering genius to make everything clear? For years startups, like the frenetically inventive
Introduction: "But it is not complicated. [There's] just a lot of it." \--Richard Feynmanon how the immense variety of the world arises from simple rules.Contents:Have We Reached the End of Scaling?Applications Become Black Boxes Using Markets to Scale and Control CostsLet's Welcome our Neo-Feudal OverlordsThe Economic Argument for the Ambient CloudWhat Will Kill the Cloud?The Amazing Collective Compute Power of the Ambient CloudUsing the Ambient Cloud as an Application RuntimeApplications as Virtual StatesConclusionWe have not yet begun to scale. The world is still fundamentally disconnected and for all our wisdom we are still in the earliest days of learning how to build truly large planet-scaling applications.Today 350 million users on Facebook is a lot of users and five million followers on Twitter is a lot of followers. This may seem like a lot now, but consider we have no planet wide applications yet. None.Tomorrow the numbers foreshadow a newCambrian explosionof connectivity that will look as
Introduction: All in all this is still my favorite post and I still think it's an accurate vision of a future. Not everyone agrees, but I guess we'll see..."But it is not complicated. [There's] just a lot of it." \--Richard Feynmanon how the immense variety of the world arises from simple rules.Contents:Have We Reached the End of Scaling?Applications Become Black Boxes Using Markets to Scale and Control CostsLet's Welcome our Neo-Feudal OverlordsThe Economic Argument for the Ambient CloudWhat Will Kill the Cloud?The Amazing Collective Compute Power of the Ambient CloudUsing the Ambient Cloud as an Application RuntimeApplications as Virtual StatesConclusionWe have not yet begun to scale. The world is still fundamentally disconnected and for all our wisdom we are still in the earliest days of learning how to build truly large planet-scaling applications.Today 350 million users on Facebook is a lot of users and five million followers on Twitter is a lot of followers. This may seem like a lot now, but c
6 0.12732829 96 high scalability-2007-09-18-Amazon Architecture
7 0.11383105 33 high scalability-2007-07-26-ThemBid Architecture
8 0.11250366 202 high scalability-2008-01-06-Email Architecture
9 0.11025742 910 high scalability-2010-09-30-Facebook and Site Failures Caused by Complex, Weakly Interacting, Layered Systems
10 0.10833742 954 high scalability-2010-12-06-What the heck are you actually using NoSQL for?
11 0.10819221 129 high scalability-2007-10-23-Hire Facebook, Ning, and Salesforce to Scale for You
12 0.10757085 1444 high scalability-2013-04-23-Facebook Secrets of Web Performance
13 0.10722313 1507 high scalability-2013-08-26-Reddit: Lessons Learned from Mistakes Made Scaling to 1 Billion Pageviews a Month
14 0.10683429 1520 high scalability-2013-09-20-Stuff The Internet Says On Scalability For September 20, 2013
16 0.10353451 1011 high scalability-2011-03-25-Did the Microsoft Stack Kill MySpace?
17 0.101339 870 high scalability-2010-08-02-7 Scaling Strategies Facebook Used to Grow to 500 Million Users
18 0.09960667 920 high scalability-2010-10-15-Troubles with Sharding - What can we learn from the Foursquare Incident?
20 0.097554907 863 high scalability-2010-07-22-How can we spark the movement of research out of the Ivory Tower and into production?
topicId topicWeight
[(0, 0.215), (1, 0.083), (2, -0.004), (3, 0.017), (4, 0.071), (5, -0.091), (6, -0.028), (7, 0.041), (8, 0.004), (9, -0.053), (10, -0.067), (11, 0.064), (12, -0.009), (13, 0.006), (14, 0.106), (15, -0.091), (16, 0.011), (17, 0.003), (18, -0.041), (19, 0.035), (20, -0.005), (21, -0.038), (22, 0.036), (23, 0.015), (24, -0.045), (25, 0.024), (26, 0.019), (27, -0.006), (28, -0.044), (29, 0.02), (30, -0.058), (31, 0.028), (32, -0.022), (33, 0.005), (34, -0.006), (35, -0.0), (36, -0.035), (37, 0.003), (38, 0.052), (39, 0.009), (40, -0.033), (41, 0.042), (42, 0.044), (43, -0.003), (44, -0.005), (45, -0.005), (46, 0.002), (47, -0.054), (48, 0.029), (49, -0.023)]
simIndex simValue blogId blogTitle
same-blog 1 0.98989451 1492 high scalability-2013-07-17-How do you create a 100th Monkey software development culture?
Introduction: Someone reading my now ancient C++ coding standard recommendation for using doxygen to automatically generate documentation from source code, asked a great question: I've often considered using doxygen, I always ask myself - is this really useful? Would I use it if I were new to a project? Would programmers working on the project use it? I'll rephrase their question to more conveniently express a point I've thought a lot about: Why do companies put so little effort into automating their own development process to make development easier? It's like the hair stylist whose own hair looks like someone cut it using a late night infomercial vacuum cleaner attachment. Or it's like the interior decorator whose own house looks like a monk's cell. Software organizations rarely build software to make developing software easier. Why is that? Because there are three ways changes are made in an organization: Top Dog - Someone so high up in the org chart you would need
2 0.91193414 1500 high scalability-2013-08-12-100 Curse Free Lessons from Gordon Ramsay on Building Great Software
Introduction: Gordon Ramsay is a world renowned chef with a surprising amount to say on software development. Well, he says it about cooking and running a restaurant, but it applies to software development too. You may have seen Gordon Ramsay on one of his many TV shows. Hell's Kitchen is a competition between chefs trying to win a dream job: head chef of their own high-end restaurant. On this show Ramsay is judge, jury, and executioner. And he chops off more than a few heads. Kitchen Nightmares is a show where Ramsay is called in by restaurant owners to help turn around their failing restaurants. On this show Ramsay is there to help. If you just watch Hell's Kitchen you will likely conclude Ramsay is one of the devil's own helpers ("ram" is the symbol of the devil and "say" means he speaks for the devil: Ramsay). Ramsay screams, yells, cusses, belittles, and throws tantrums even a 7 year old could learn from. Then he does it all over gain just for spite. In Hell's Kitchen there's no evidence a
3 0.90409577 347 high scalability-2008-07-07-Five Ways to Stop Framework Fixation from Crashing Your Scaling Strategy
Introduction: If you've wondered why I haven't been posting lately it's because I've been on an amazing Beach's motorcycle tour of the Alps ( and , and , and , and , and , and , and , and ). My wife (Linda) and I rode two-up on a BMW 1200 GS through the alps in Germany, Austria, Switzerland, Italy, Slovenia, and Lichtenstein. The trip was more beautiful than I ever imagined. We rode challenging mountain pass after mountain pass, froze in the rain, baked in the heat, woke up on excellent Italian coffee, ate slice after slice of tasty apple strudel, drank dazzling local wines, smelled the fresh cut grass as the Swiss en masse cut hay for the winter feeding of their dairy cows, rode the amazing Munich train system, listened as cow bells tinkled like wind chimes throughout small valleys, drank water from a pure alpine spring on a blisteringly hot hike, watched local German folk dancers represent their regions, and had fun in the company of fellow riders. Magical. They say you'll ride more
4 0.88709927 863 high scalability-2010-07-22-How can we spark the movement of research out of the Ivory Tower and into production?
Introduction: Over the years I've read a lot of research papers looking for better ways of doing things. Sometimes I find ideas I can use, but more often than not I come up empty. The problem is there are very few good papers. And by good I mean: can a reasonably intelligent person read a paper and turn it into something useful? Now, clearly I'm not an academic and clearly I'm no genius, I'm just an everyday programmer searching for leverage, and as a common specimen of the species I've often thought how much better our industry would be if we could simply move research from academia into production with some sort of self-conscious professionalism. Currently the process is horribly hit or miss. And this problem extends equally to companies with research divisions that often do very little to help front-line developers succeed. How many ideas break out of academia into industry in computer science? We have many brilliant examples: encryption, microprocessors, compression, transactions, distribu
5 0.87962699 1503 high scalability-2013-08-19-What can the Amazing Race to the South Pole Teach us About Startups?
Introduction: At the heart of every software adventure exists a journey in service of a quest. Melodramatic much? Sorry, but while wandering dazzled through Race to the End of the Earth , a fantastic exhibit at the Royal BC Museum on the 1911-1912 race to the South Pole between Norwegian explorer Roald Amundsen and British naval officer Robert Scott , I couldn’t help but think of the two radically different approaches each team took to the race and it shocked me to see that some of the same principles that lead to success or failure in software development also seem to lead to success or failure in exploration. I wish I could reproduce the experience of walking through the exhibit . Plaque after plaque I remember wondering out loud at Scott’s choices and then nod in agreement with Amundsen’s approach. The core conflict was straight out of any ancient Agile (Amundsen) vs Waterfall (Scott) thread you can find on Usenet. And Waterfall lost. As background here are some sources you may want
6 0.8779614 352 high scalability-2008-07-18-Robert Scoble's Rules for Successfully Scaling Startups
7 0.87673455 1388 high scalability-2013-01-16-What if Cars Were Rented Like We Hire Programmers?
8 0.86377341 1635 high scalability-2014-04-21-This is why Microsoft won. And why they lost.
10 0.85086018 1379 high scalability-2012-12-31-Designing for Resiliency will be so 2013
11 0.84392798 1477 high scalability-2013-06-18-Scaling Mailbox - From 0 to One Million Users in 6 Weeks and 100 Million Messages Per Day
12 0.83003515 1366 high scalability-2012-12-03-Resiliency is the New Normal - A Deep Look at What It Means and How to Build It
13 0.82786644 1012 high scalability-2011-03-28-Aztec Empire Strategy: Use Dual Pipes in Your Aqueduct for High Availability
14 0.82473326 1410 high scalability-2013-02-20-Smart Companies Fail Because they Do Everything Right - Staying Alive to Scale
16 0.81828225 1377 high scalability-2012-12-26-Ask HS: What will programming and architecture look like in 2020?
17 0.80603188 1011 high scalability-2011-03-25-Did the Microsoft Stack Kill MySpace?
18 0.80524915 1014 high scalability-2011-03-31-8 Lessons We Can Learn from the MySpace Incident - Balance, Vision, Fearlessness
19 0.80460232 1422 high scalability-2013-03-12-If Your System was a Symphony it Might Sound Like This...
20 0.80407339 1636 high scalability-2014-04-23-Here's a 1300 Year Old Solution to Resilience - Rebuild, Rebuild, Rebuild
topicId topicWeight
[(1, 0.14), (2, 0.215), (10, 0.039), (30, 0.025), (40, 0.231), (61, 0.081), (77, 0.013), (79, 0.112), (85, 0.016), (94, 0.05)]
simIndex simValue blogId blogTitle
1 0.98560399 1419 high scalability-2013-03-07-It's a VM Wasteland - A Near Optimal Packing of VMs to Machines Reduces TCO by 22%
Introduction: In Algorithm Design for Performance Aware VM Consolidation we learn some shocking facts (gambling in Casablanca?): Average server utilization in many data centers is low, estimated between 5% and 15%. This is wasteful because an idle server often consumes more than 50% of peak power. Surely that's just for old style datacenters? Nope. In Google data centers, workloads that are consolidated use only 50% of the processor cores. Every other processor core is left unused simply to ensure that performance does not degrade. It's a VM wasteland. The goal is to reduce waste by packing VMs onto machines without hurting performance or wasting resources. The idea is to select VMs that interfere the least with each other and places them together on the same server. It's a NP-Complete problem, but this paper describes a practical method that performs provably close to the optimal. Interestingly they can optimize for performance or power efficiency, so you can use different algorithm
2 0.982346 860 high scalability-2010-07-17-Hot Scalability Links for July 17, 2010
Introduction: And by hot I also mean temperature. Summer has arrived. It's sizzling here in Silicon Valley. Thank you air conditioning! Scale the web by appointing a Crawler Czar? Tom Foremski has the idea that Google should open up their index so sites wouldn't have to endure the constant pounding by ravenous crawler bots. Don MacAskill of SmugMug estimates 50% of our web server CPU resources are spent serving crawlers. What a waste. How this would all work with real-time feeds, paid feeds (Twitter, movies, ...), etc. is unknown, but does it make sense for all that money to be spent on extracting the same data over and over again? Tweets of Gold: jamesurquhart : Key to applications is architecture. Key for infrastructure supporting archs is configurability. Configurability==features . tjake : People who choose their datastore based oh hearsay and not their own evaluation are doomed . b6n : No global lock ever goes unpunished. MichaelSurt
Introduction: While working with Memcache the other night, it dawned on me that it’s usage as a distributed caching mechanism was really just one of many ways to use it. That there are in fact many alternative usages that one could find for Memcache if they could just realize what Memcache really is at its core – a simple distributed hash-table – is an important point worthy of further discussion. To be clear, when I say “simple”, by no means am I implying that Memcache’s implementation is simple, just that the ideas behind it are such. Think about that for a minute. What else could we use a simple distributed hash-table for, besides caching? How about using it as an alternative to the traditional shard lookup method we used in our Master Index Lookup scalability strategy, discussed previously here.
4 0.96769267 778 high scalability-2010-02-15-The Amazing Collective Compute Power of the Ambient Cloud
Introduction: This is an excerpt from my article Building Super Scalable Systems: Blade Runner Meets Autonomic Computing in the Ambient Cloud. Earlier we talked about how a single botnet could harness more compute power than our largest super computers. Well, that's just the start of it. The amount of computer power available to the Ambient Cloud will be truly astounding. 2 Billion Personal Computers The number of personal computers is still growing. By 2014 one estimate is there will be 2 billion PCs . That's a giant reservoir of power to exploit, especially considering these new boxes are stuffed with multiple powerful processors and gigabytes of memory. 7 Billion Smartphones By now it's common wisdom smartphones are the computing platform of the future. It's plausible to assume the total number of mobile phones in use will roughly equal the number of people on earth. That's 7 billion smartphones. Smartphones aren't just tiny little wannabe computers anymore either. They are rea
5 0.96500146 330 high scalability-2008-05-27-Should Twitter be an All-You-Can-Eat Buffet or a Vending Machine?
Introduction: Om proposes one solution to the Twitter Problem is to limit followers to three square meals a day. The reasonable idea being that lower limits should mean fewer scaling problems. And as a kicker raising those limits is a good way to raise much needed revenue. Scoble thinks users should consume without limit and will drive to another buffet if all-you-can-eat privileges are revoked. The reasonable idea being that if an internet service can't solve internet scale problems then there's not much use for it. Dave says comp power users a top floor suite and shower them with free passes to the buffet. Let the good times roll! The reasonable idea being that power users help create popular restaurants, er, services in the first place and limiting them starves users and starved users won't come back. So, should web services like Twitter be a buffet, a fixed eight course fine dining experience, a small plate restaurant, a family style joint, or a vending machine? Or
6 0.95971626 1466 high scalability-2013-05-29-Amazon: Creating a Customer Utopia One Culture Hack at a Time
7 0.94619471 848 high scalability-2010-06-25-Hot Scalability Links for June 25, 2010
8 0.94323361 1414 high scalability-2013-03-01-Stuff The Internet Says On Scalability For February 29, 2013
9 0.93899632 1471 high scalability-2013-06-06-Paper: Memory Barriers: a Hardware View for Software Hackers
10 0.93717992 300 high scalability-2008-04-07-Scalr - Open Source Auto-scaling Hosting on Amazon EC2
12 0.93487978 879 high scalability-2010-08-12-Think of Latency as a Pseudo-permanent Network Partition
13 0.93312711 768 high scalability-2010-02-01-What Will Kill the Cloud?
14 0.92523038 402 high scalability-2008-10-05-Paper: Scalability Design Patterns
15 0.92385256 757 high scalability-2010-01-04-11 Strategies to Rock Your Startup’s Scalability in 2010
same-blog 16 0.91417611 1492 high scalability-2013-07-17-How do you create a 100th Monkey software development culture?
17 0.91155893 27 high scalability-2007-07-25-Product: 3 PAR REMOTE COPY
18 0.87625003 338 high scalability-2008-06-02-Total Cost of Ownership for different web development frameworks
19 0.8621605 97 high scalability-2007-09-18-Session management in highly scalable web sites
20 0.85484141 19 high scalability-2007-07-16-Paper: Replication Under Scalable Hashing