high_scalability high_scalability-2013 high_scalability-2013-1444 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: This is a repost of part 1 of an interview I did for the Boundary blog . Boundary: What is Facebook’s secret sauce for managing what’s got to be the biggest Big Data project, if you will, on the Web? Hoff: From several presentations we’ve learned what Facebook insiders like Aditya Agarwal and Robert Johnson , both former Directors of Engineering, consider their secret sauce: Scaling Takes Iteration . Solutions often work in the beginning, but you’ll have to modify them as you go. PHP, for example, is simple to use at first, but is not a good choice when you have tens of thousands of web servers. Scaling Takes Iteration . You can say that again. Don’t Over-Design . Just use what you need as you scale your system out. Figure out where you need to iterate on a solution, optimize something, or completely build a part of the stack yourself. Choose the Right Tool for the Job . Realize that any choice comes with overhead. If you really need to use P
sentIndex sentText sentNum sentScore
1 Boundary: What is Facebook’s secret sauce for managing what’s got to be the biggest Big Data project, if you will, on the Web? [sent-2, score-0.236]
2 Hoff: From several presentations we’ve learned what Facebook insiders like Aditya Agarwal and Robert Johnson , both former Directors of Engineering, consider their secret sauce: Scaling Takes Iteration . [sent-3, score-0.192]
3 PHP, for example, is simple to use at first, but is not a good choice when you have tens of thousands of web servers. [sent-5, score-0.112]
4 Yet with that choice there is overhead, usually across deployment, monitoring, ops, and so on. [sent-14, score-0.112]
5 Isolate the part of the culture that you value and want to preserve. [sent-18, score-0.11]
6 It brings the site down regularly (out of memory, infinite loops), but there’s a big potential payoff as they figure out how to make it work. [sent-25, score-0.299]
7 Get the right set of people, empower them and let them work. [sent-29, score-0.099]
8 The best tools for scaling are an engineering and operations teams that can handle anything. [sent-32, score-0.186]
9 Production is where the really useful data comes from. [sent-36, score-0.099]
10 If a team is responsible for something they must also control it. [sent-40, score-0.169]
11 You can’t move fast unless you have small teams who have control and responsibility. [sent-42, score-0.663]
12 You can’t know how your changes are working unless you get those changes into production and measure results. [sent-43, score-0.425]
13 You can’t move code into production unless people feel responsible for moving out working code. [sent-44, score-0.673]
14 You can’t handle the scale unless you figure out how to scale horizontally, move fast and measure everything– that all comes down to good people. [sent-45, score-0.653]
15 We see a blast wave of innovation coming from technological hotspots like Google, Netflix, Twitter and Facebook. [sent-49, score-0.183]
16 People are constantly willing to share their experiences, share their code and talk about what works. [sent-52, score-0.226]
17 My wife is a tax accountant and they definitely don’t have the same vibe which is a little sad. [sent-53, score-0.198]
18 There are a lot of unbelievably smart and passionate people in this field and total quality only rises the more people talk about how to build great stuff. [sent-54, score-0.587]
19 It’s also pretty obvious to me that having a quality site and willingness to share are linked. [sent-55, score-0.385]
20 There are literally hundreds of others that could be mentioned, but these companies have continually and enthusiastically contributed to advancing the state of the art in Web performance. [sent-58, score-0.27]
wordName wordTfidf (topN-words)
[('unless', 0.215), ('teams', 0.186), ('hoff', 0.171), ('hiphop', 0.171), ('facebook', 0.148), ('sauce', 0.136), ('figure', 0.126), ('measure', 0.124), ('share', 0.113), ('choice', 0.112), ('people', 0.11), ('directors', 0.11), ('interviewi', 0.11), ('theboundary', 0.11), ('culture', 0.11), ('enthusiastically', 0.103), ('unbelievably', 0.103), ('rises', 0.103), ('secret', 0.1), ('comes', 0.099), ('obvious', 0.099), ('hotspots', 0.099), ('wife', 0.099), ('empower', 0.099), ('vibe', 0.099), ('payoff', 0.095), ('small', 0.094), ('insiders', 0.092), ('innovating', 0.092), ('promotes', 0.092), ('willingness', 0.092), ('ca', 0.092), ('responsible', 0.09), ('prismatic', 0.089), ('repost', 0.089), ('companies', 0.089), ('move', 0.089), ('production', 0.086), ('netflix', 0.086), ('virtuous', 0.085), ('blast', 0.084), ('contributors', 0.084), ('feel', 0.083), ('johnson', 0.082), ('tripadvisor', 0.081), ('quality', 0.081), ('great', 0.08), ('control', 0.079), ('regularly', 0.078), ('contributed', 0.078)]
simIndex simValue blogId blogTitle
same-blog 1 1.0 1444 high scalability-2013-04-23-Facebook Secrets of Web Performance
Introduction: This is a repost of part 1 of an interview I did for the Boundary blog . Boundary: What is Facebook’s secret sauce for managing what’s got to be the biggest Big Data project, if you will, on the Web? Hoff: From several presentations we’ve learned what Facebook insiders like Aditya Agarwal and Robert Johnson , both former Directors of Engineering, consider their secret sauce: Scaling Takes Iteration . Solutions often work in the beginning, but you’ll have to modify them as you go. PHP, for example, is simple to use at first, but is not a good choice when you have tens of thousands of web servers. Scaling Takes Iteration . You can say that again. Don’t Over-Design . Just use what you need as you scale your system out. Figure out where you need to iterate on a solution, optimize something, or completely build a part of the stack yourself. Choose the Right Tool for the Job . Realize that any choice comes with overhead. If you really need to use P
2 0.41581663 840 high scalability-2010-06-10-The Four Meta Secrets of Scaling at Facebook
Introduction: Aditya Agarwal, Director of Engineering at Facebook, gave an excellent Scale at Facebook talk that covers their architecture, but the talk is really more about how to scale an organization by preserving the best parts of its culture. The key take home of the talk is: You can get the code right, you can get the products right, but you need to get the culture right first. If you don't get the culture right then your company won't scale. This leads into the four meta secrets of scaling at Facebook: Scaling takes Iteration Don't Over Design Choose the right tool for the job, but realize that your choice comes with overhead. Get the culture right. Move Fast - break things. Huge Impact - small teams. Be bold - innovate. Some Background Facebook is big : 400 million active users; users spend an average of 20 minutes a day; 5 billion pieces of content (status updates, comments, likes, photo uploads, video uploads, chat messages, inbox messages, group events, f
3 0.35313079 870 high scalability-2010-08-02-7 Scaling Strategies Facebook Used to Grow to 500 Million Users
Introduction: Robert Johnson, a director of engineering at Facebook, celebrated Facebook's monumental achievement of reaching 500 million users by sharing the scaling principles that helped reach that milestone. In case you weren't suitably impressed by the 500 million user number, Robert ratchets up the numbers game with these impressive figures: 1 million users per engineer 500 million active users 100 billion hits per day 50 billion photos 2 trillion objects cached, with hundreds of millions of requests per second 130TB of logs every day How did Facebook get to this point? People Matter Most . It's people who build and run systems. The best tools for scaling are an engineering and operations teams that can handle anything. Scale Horizontally . Handling exponentially growing traffic requires spreading load arbitrarily across many machines. Using different databases for tables like accounts and profiles only doubles capacity. This approach hurts efficiency, but
4 0.19681603 1240 high scalability-2012-05-07-Startups are Creating a New System of the World for IT
Introduction: It remains that, from the same principles, I now demonstrate the frame of the System of the World. -- Isaac Newton The practice of IT reminds me a lot of the practice of science before Isaac Newton. Aristotelianism was dead, but there was nothing to replace it. Then Newton came along, created a scientific revolution with his System of the World . And everything changed. That was New System of the World number one. New System of the World number two was written about by the incomparable Neal Stephenson in his incredible Baroque Cycle series. It explores the singular creation of a new way of organizing society grounded in new modes of thought in business, religion, politics, and science. Our modern world emerged Enlightened as it could from this roiling cauldron of forces. In IT we may have had a Leonardo da Vinci or even a Galileo, but we’ve never had our Newton. Maybe we don't need a towering genius to make everything clear? For years startups, like the frenetically inventive
5 0.15402824 1068 high scalability-2011-06-27-TripAdvisor Architecture - 40M Visitors, 200M Dynamic Page Views, 30TB Data
Introduction: This is a guest post by Andy Gelfond , VP of Engineering for TripAdvisor. Andy has been with TripAdvisor for six and a half years, wrote a lot of code in the earlier days, and has been building and running a first class engineering and operations team that is responsible for the worlds largest travel site. There's an update for this article at An Epic TripAdvisor Update: Why Not Run On The Cloud? The Grand Experiment . For TripAdvisor , scalability is woven into our organization on many levels - data center, software architecture, development/deployment/operations, and, most importantly, within the culture and organization. It is not enough to have a scalable data center, or a scalable software architecture. The process of designing, coding, testing, and deploying code also needs to be scalable. All of this starts with hiring and a culture and an organization that values and supports a distributed, fast, and effective development and operation of a complex and highly scalable co
6 0.14453207 96 high scalability-2007-09-18-Amazon Architecture
9 0.14060348 721 high scalability-2009-10-13-Why are Facebook, Digg, and Twitter so hard to scale?
10 0.13572559 1228 high scalability-2012-04-16-Instagram Architecture Update: What’s new with Instagram?
11 0.1341151 1404 high scalability-2013-02-11-At Scale Even Little Wins Pay Off Big - Google and Facebook Examples
12 0.13143809 837 high scalability-2010-06-07-Six Ways Twitter May Reach its Big Hairy Audacious Goal of One Billion Users
13 0.12552047 1359 high scalability-2012-11-15-Gone Fishin': Justin.Tv's Live Video Broadcasting Architecture
14 0.12520297 796 high scalability-2010-03-16-Justin.tv's Live Video Broadcasting Architecture
15 0.12468855 976 high scalability-2011-01-20-75% Chance of Scale - Leveraging the New Scaleogenic Environment for Growth
17 0.1233661 1011 high scalability-2011-03-25-Did the Microsoft Stack Kill MySpace?
18 0.12330779 1561 high scalability-2013-12-09-Site Moves from PHP to Facebook's HipHop, Now Pages Load in .6 Seconds Instead of Five
20 0.12129576 920 high scalability-2010-10-15-Troubles with Sharding - What can we learn from the Foursquare Incident?
topicId topicWeight
[(0, 0.253), (1, 0.071), (2, 0.019), (3, 0.004), (4, 0.099), (5, -0.136), (6, -0.081), (7, 0.078), (8, 0.054), (9, -0.024), (10, -0.02), (11, 0.08), (12, 0.007), (13, 0.012), (14, -0.003), (15, -0.01), (16, 0.04), (17, -0.023), (18, -0.003), (19, 0.091), (20, 0.043), (21, 0.052), (22, 0.096), (23, 0.005), (24, -0.024), (25, -0.02), (26, 0.03), (27, -0.009), (28, 0.044), (29, 0.048), (30, -0.113), (31, 0.013), (32, 0.027), (33, -0.001), (34, -0.029), (35, -0.011), (36, 0.013), (37, -0.09), (38, -0.023), (39, 0.014), (40, -0.044), (41, -0.014), (42, 0.061), (43, -0.062), (44, -0.043), (45, -0.039), (46, 0.004), (47, -0.048), (48, 0.06), (49, -0.035)]
simIndex simValue blogId blogTitle
same-blog 1 0.97248048 1444 high scalability-2013-04-23-Facebook Secrets of Web Performance
Introduction: This is a repost of part 1 of an interview I did for the Boundary blog . Boundary: What is Facebook’s secret sauce for managing what’s got to be the biggest Big Data project, if you will, on the Web? Hoff: From several presentations we’ve learned what Facebook insiders like Aditya Agarwal and Robert Johnson , both former Directors of Engineering, consider their secret sauce: Scaling Takes Iteration . Solutions often work in the beginning, but you’ll have to modify them as you go. PHP, for example, is simple to use at first, but is not a good choice when you have tens of thousands of web servers. Scaling Takes Iteration . You can say that again. Don’t Over-Design . Just use what you need as you scale your system out. Figure out where you need to iterate on a solution, optimize something, or completely build a part of the stack yourself. Choose the Right Tool for the Job . Realize that any choice comes with overhead. If you really need to use P
2 0.89504808 870 high scalability-2010-08-02-7 Scaling Strategies Facebook Used to Grow to 500 Million Users
Introduction: Robert Johnson, a director of engineering at Facebook, celebrated Facebook's monumental achievement of reaching 500 million users by sharing the scaling principles that helped reach that milestone. In case you weren't suitably impressed by the 500 million user number, Robert ratchets up the numbers game with these impressive figures: 1 million users per engineer 500 million active users 100 billion hits per day 50 billion photos 2 trillion objects cached, with hundreds of millions of requests per second 130TB of logs every day How did Facebook get to this point? People Matter Most . It's people who build and run systems. The best tools for scaling are an engineering and operations teams that can handle anything. Scale Horizontally . Handling exponentially growing traffic requires spreading load arbitrarily across many machines. Using different databases for tables like accounts and profiles only doubles capacity. This approach hurts efficiency, but
3 0.87829757 840 high scalability-2010-06-10-The Four Meta Secrets of Scaling at Facebook
Introduction: Aditya Agarwal, Director of Engineering at Facebook, gave an excellent Scale at Facebook talk that covers their architecture, but the talk is really more about how to scale an organization by preserving the best parts of its culture. The key take home of the talk is: You can get the code right, you can get the products right, but you need to get the culture right first. If you don't get the culture right then your company won't scale. This leads into the four meta secrets of scaling at Facebook: Scaling takes Iteration Don't Over Design Choose the right tool for the job, but realize that your choice comes with overhead. Get the culture right. Move Fast - break things. Huge Impact - small teams. Be bold - innovate. Some Background Facebook is big : 400 million active users; users spend an average of 20 minutes a day; 5 billion pieces of content (status updates, comments, likes, photo uploads, video uploads, chat messages, inbox messages, group events, f
4 0.85017604 378 high scalability-2008-09-03-Some Facebook Secrets to Better Operations
Introduction: Kim Nash in an interview with Jonathan Heiliger , Facebook VP of technical operations, provides some juicy details on how Facebook handles operations. Operations is one of those departments everyone runs differently as it is usually an ontogeny recapitulates phylogeny situation. With 2,000 databases, 25 terabytes of cache, 90 million active users, and 10,000 servers you know Facebook has some serious operational issues. What are some of Facebook's secrets to better operations? Frequent Releases . A major release once a week and a minor releases every few days. Create a Cyber Liability Group . At one time operations was distributed amongst several groups. A permanent operations group was created to isolate problems and revert problem software components back to previously known good states. The ability of a separate team to handle rollbacks speaks to a great deal of standardization and advanced tool building. Distribute Team Across Time Zones . Split the operations team ac
Introduction: Facebook has been teasing us. While many of their recent acquisitions have been surprising, shocking is the only word adequately describing Facebook's 5 day whirlwind acquisition of Oculus , immersive virtual reality visionaries, for a now paltry sounding $2 billion. The backlash is a pandemic, jumping across social networks with the speed only a meme powered by the directly unaffected can generate. For more than 30 years VR has been the dream burning in the heart of every science fiction fan. Now that this future might finally be here, Facebook’s ownage makes it seem like a wonderful and hopeful timeline has been choked off, killing the Metaverse before it even had a chance to begin. For the many who voted for an open future with their Kickstarter dollars , there’s a deep and personal sense of betrayal, despite Facebook’s promise to leave Oculus alone. The intensity of the reaction is because Oculus matters to people. It's new, it's different, it create
6 0.78195632 129 high scalability-2007-10-23-Hire Facebook, Ning, and Salesforce to Scale for You
7 0.78185797 966 high scalability-2010-12-31-Facebook in 20 Minutes: 2.7M Photos, 10.2M Comments, 4.6M Messages
8 0.76042265 1011 high scalability-2011-03-25-Did the Microsoft Stack Kill MySpace?
9 0.7435199 1502 high scalability-2013-08-16-Stuff The Internet Says On Scalability For August 16, 2013
10 0.7360667 1323 high scalability-2012-09-15-4 Reasons Facebook Dumped HTML5 and Went Native
11 0.73172033 1492 high scalability-2013-07-17-How do you create a 100th Monkey software development culture?
12 0.72961223 1081 high scalability-2011-07-18-Building your own Facebook Realtime Analytics System
13 0.72818643 1168 high scalability-2012-01-04-How Facebook Handled the New Year's Eve Onslaught
14 0.71883261 264 high scalability-2008-03-03-Read This Site and Ace Your Next Interview!
15 0.71761811 1500 high scalability-2013-08-12-100 Curse Free Lessons from Gordon Ramsay on Building Great Software
16 0.71662855 1458 high scalability-2013-05-15-Lesson from Airbnb: Give Yourself Permission to Experiment with Non-scalable Changes
18 0.70650709 1014 high scalability-2011-03-31-8 Lessons We Can Learn from the MySpace Incident - Balance, Vision, Fearlessness
19 0.70419645 1617 high scalability-2014-03-21-Stuff The Internet Says On Scalability For March 21st, 2014
20 0.70311785 1171 high scalability-2012-01-09-The Etsy Saga: From Silos to Happy to Billions of Pageviews a Month
topicId topicWeight
[(1, 0.177), (2, 0.223), (10, 0.069), (28, 0.035), (37, 0.132), (40, 0.027), (61, 0.09), (79, 0.13), (85, 0.05)]
simIndex simValue blogId blogTitle
1 0.97269303 1029 high scalability-2011-04-25-The Big List of Articles on the Amazon Outage
Introduction: Please see The Updated Big List Of Articles On The Amazon Outage for a new improved list. So many great articles have been written on the Amazon Outage. Some aim at being helpful, some chastise developers for being so stupid, some chastise Amazon for being so incompetent, some talk about the pain they and their companies have experienced, and some even predict the downfall of the cloud. Still others say we have seen a sea change in future of the cloud, a prediction that's hard to disagree with, though the shape of the change remains...cloudy. I'll try to keep this list update as more information comes out. There will be a lot for developers to consider going forward. If there's a resource you think should be added, just let me know. Amazon's Explanation of What Happened Summary of the Amazon EC2 and Amazon RDS Service Disruption in the US East Region Hackers News thread on AWS Service Disruption Post Mortem Quite Funny Commentary on the Summary Experiences f
2 0.95710373 1133 high scalability-2011-10-27-Strategy: Survive a Comet Strike in the East With Reserved Instances in the West
Introduction: Ali Sadat of MuleSoft gave interesting presentation at Saturday's Talk Cloudy to Me! event about their experiences moving Mule iON , their ESB (enterprise service bus) product to the cloud. First, a little about Talk Cloudy to Me . This event is the second one day cloud event put on by Sebastian Stadil, founder of Scalr, as part of the Cloud Computing Meetup group, also created by Sebastian. Sebastian has become a master at running these mini-conference style events. Really a quality job by him and his dedicated crew. These events are free, sponsored by various vendors; they are short, 11-5; the food is good, Thai; the venue is nice, eBay; they are on topic, with cloud and other speakers giving 30-45 minute talks. More on the event when the video comes out. I type as fast as I can but I can't do much without the video. Back to Ali Sadat. While he gave a lot of lessons--the integration will your billing system will take a lot longer than you think; talk to your Amazon accou
same-blog 3 0.95044661 1444 high scalability-2013-04-23-Facebook Secrets of Web Performance
Introduction: This is a repost of part 1 of an interview I did for the Boundary blog . Boundary: What is Facebook’s secret sauce for managing what’s got to be the biggest Big Data project, if you will, on the Web? Hoff: From several presentations we’ve learned what Facebook insiders like Aditya Agarwal and Robert Johnson , both former Directors of Engineering, consider their secret sauce: Scaling Takes Iteration . Solutions often work in the beginning, but you’ll have to modify them as you go. PHP, for example, is simple to use at first, but is not a good choice when you have tens of thousands of web servers. Scaling Takes Iteration . You can say that again. Don’t Over-Design . Just use what you need as you scale your system out. Figure out where you need to iterate on a solution, optimize something, or completely build a part of the stack yourself. Choose the Right Tool for the Job . Realize that any choice comes with overhead. If you really need to use P
4 0.95024955 1033 high scalability-2011-05-02-The Updated Big List of Articles on the Amazon Outage
Introduction: Since The Big List Of Articles On The Amazon Outage was published we've a had few updates that people might not have seen. Amazon of course released their Summary of the Amazon EC2 and Amazon RDS Service Disruption in the US East Region . Netlix shared their Lessons Learned from the AWS Outage as did Heroku ( How Heroku Survived the Amazon Outage ), Smug Mug ( How SmugMug survived the Amazonpocalypse ), and SimpleGeo ( How SimpleGeo Stayed Up During the AWS Downtime ). The curious thing from my perspective is the general lack of response to Amazon's explanation. I expected more discussion. There's been almost none that I've seen. My guess is very few people understand what Amazon was talking about enough to comment whereas almost everyone feels qualified to talk about the event itself. Lesson for crisis handlers : deep dive post-mortems that are timely, long, honestish, and highly technical are the most effective means of staunching the downward spiral of media attention.
5 0.94162023 103 high scalability-2007-09-28-Kosmos File System (KFS) is a New High End Google File System Option
Introduction: There's a new clustered file system on the spindle: Kosmos File System (KFS) . Thanks to Rich Skrenta for turning me on to KFS and I think his blog post says it all. KFS is an open source project written in C++ by search startup Kosmix . The team members have a good pedigree so there's a better than average chance this software will be worth considering. After you stop trying to turn KFS into "Kentucky Fried File System" in your mind, take a look at KFS' intriguing feature set: Incremental scalability: New chunkserver nodes can be added as storage needs increase; the system automatically adapts to the new nodes. Availability: Replication is used to provide availability due to chunk server failures. Typically, files are replicated 3-way. Per file degree of replication: The degree of replication is configurable on a per file basis, with a max. limit of 64. Re-replication: Whenever the degree of replication for a file drops below the configured amount (
6 0.93977326 965 high scalability-2010-12-29-Pinboard.in Architecture - Pay to Play to Keep a System Small
7 0.93408537 279 high scalability-2008-03-17-Microsoft's New Database Cloud Ready to Rumble with Amazon
8 0.9303984 1379 high scalability-2012-12-31-Designing for Resiliency will be so 2013
9 0.92519265 311 high scalability-2008-04-29-Strategy: Sample to Reduce Data Set
10 0.92329115 314 high scalability-2008-05-03-Product: nginx
11 0.91938907 1366 high scalability-2012-12-03-Resiliency is the New Normal - A Deep Look at What It Means and How to Build It
12 0.91650003 1041 high scalability-2011-05-15-Building a Database remote availability site
13 0.91622722 340 high scalability-2008-06-06-Economies of Non-Scale
14 0.91613019 1118 high scalability-2011-09-19-Big Iron Returns with BigMemory
15 0.91301394 1506 high scalability-2013-08-23-Stuff The Internet Says On Scalability For August 23, 2013
16 0.91295344 1 high scalability-2007-07-06-Start Here
17 0.91248649 857 high scalability-2010-07-13-DbShards Part Deux - The Internals
18 0.91099751 1008 high scalability-2011-03-22-Facebook's New Realtime Analytics System: HBase to Process 20 Billion Events Per Day
19 0.91079068 498 high scalability-2009-01-20-Product: Amazon's SimpleDB
20 0.91049445 1626 high scalability-2014-04-04-Stuff The Internet Says On Scalability For April 4th, 2014