high_scalability high_scalability-2009 high_scalability-2009-562 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Facebook's engg. director aditya talks about facebook architecture. How they use mysql, php and memcache. How they have modified the above to suit their requirements.
sentIndex sentText sentNum sentScore
1 How they have modified the above to suit their requirements. [sent-4, score-0.886]
wordName wordTfidf (topN-words)
[('suit', 0.551), ('aditya', 0.518), ('modified', 0.335), ('director', 0.301), ('facebook', 0.298), ('talks', 0.214), ('requirements', 0.197), ('php', 0.186), ('mysql', 0.12), ('use', 0.049)]
simIndex simValue blogId blogTitle
same-blog 1 1.0 562 high scalability-2009-04-10-Facebook's Aditya giving presentation on Facebook Architecture
Introduction: Facebook's engg. director aditya talks about facebook architecture. How they use mysql, php and memcache. How they have modified the above to suit their requirements.
2 0.16221417 845 high scalability-2010-06-22-Exploring the software behind Facebook, the world’s largest site
Introduction: Peter Alguacil at Pingdom wrote a HighScalability worthy article on Facebook's architecture: Exploring the software behind Facebook, the world’s largest site . It covers the challenges Facebook faces, the software Facebook uses, and the techniques Facebook uses to keep on scaling. Definitely worth a look.
3 0.15456091 624 high scalability-2009-06-10-Hive - A Petabyte Scale Data Warehouse using Hadoop
Introduction: This post about using Hive and Hadoop for analytics comes straight from Facebook engineers. Scalable analysis on large data sets has been core to the functions of a number of teams at Facebook - both engineering and non-engineering. Apart from ad hoc analysis and business intelligence applications used by analysts across the company, a number of Facebook products are also based on analytics. These products range from simple reporting applications like Insights for the Facebook Ad Network, to more advanced kind such as Facebook's Lexicon product. As a result a flexible infrastructure that caters to the needs of these diverse applications and users and that also scales up in a cost effective manner with the ever increasing amounts of data being generated on Facebook, is critical. Hive and Hadoop are the technologies that we have used to address these requirements at Facebook. Read the rest of the article on Engineering @ Facebook's Notes page
4 0.13913344 720 high scalability-2009-10-12-High Performance at Massive Scale – Lessons learned at Facebook
Introduction: Jeff Rothschild, Vice President of Technology at Facebook gave a great presentation at UC San Diego on our favorite subject: " High Performance at Massive Scale – Lessons learned at Facebook ". The abstract for the talk is: Facebook has grown into one of the largest sites on the Internet today serving over 200 billion pages per month. The nature of social data makes engineering a site for this level of scale a particularly challenging proposition. In this presentation, I will discuss the aspects of social data that present challenges for scalability and will describe the the core architectural components and design principles that Facebook has used to address these challenges. In addition, I will discuss emerging technologies that offer new opportunities for building cost-effective high performance web architectures. There's a lot of interesting about this talk that we'll get into later, but I thought you might want a head start on learning how Facebook handles 30K+ machines,
5 0.12324873 840 high scalability-2010-06-10-The Four Meta Secrets of Scaling at Facebook
Introduction: Aditya Agarwal, Director of Engineering at Facebook, gave an excellent Scale at Facebook talk that covers their architecture, but the talk is really more about how to scale an organization by preserving the best parts of its culture. The key take home of the talk is: You can get the code right, you can get the products right, but you need to get the culture right first. If you don't get the culture right then your company won't scale. This leads into the four meta secrets of scaling at Facebook: Scaling takes Iteration Don't Over Design Choose the right tool for the job, but realize that your choice comes with overhead. Get the culture right. Move Fast - break things. Huge Impact - small teams. Be bold - innovate. Some Background Facebook is big : 400 million active users; users spend an average of 20 minutes a day; 5 billion pieces of content (status updates, comments, likes, photo uploads, video uploads, chat messages, inbox messages, group events, f
6 0.11041636 264 high scalability-2008-03-03-Read This Site and Ace Your Next Interview!
7 0.10970702 1123 high scalability-2011-09-23-The Real News is Not that Facebook Serves Up 1 Trillion Pages a Month…
9 0.097105101 1561 high scalability-2013-12-09-Site Moves from PHP to Facebook's HipHop, Now Pages Load in .6 Seconds Instead of Five
10 0.08609128 943 high scalability-2010-11-16-Facebook's New Real-time Messaging System: HBase to Store 135+ Billion Messages a Month
11 0.085481435 870 high scalability-2010-08-02-7 Scaling Strategies Facebook Used to Grow to 500 Million Users
12 0.084404446 721 high scalability-2009-10-13-Why are Facebook, Digg, and Twitter so hard to scale?
13 0.081004195 792 high scalability-2010-03-10-How FarmVille Scales - The Follow-up
14 0.079379067 464 high scalability-2008-12-13-Strategy: Facebook Tweaks to Handle 6 Time as Many Memcached Requests
15 0.078943893 1081 high scalability-2011-07-18-Building your own Facebook Realtime Analytics System
16 0.076062053 196 high scalability-2007-12-30-MySQL clustering strategies and comparisions
17 0.071493797 503 high scalability-2009-01-27-Video: Storage in the Cloud at Joyent
18 0.071365409 15 high scalability-2007-07-16-Blog: MySQL Performance Blog - Everything about MySQL Performance.
19 0.068116687 378 high scalability-2008-09-03-Some Facebook Secrets to Better Operations
20 0.066900693 1619 high scalability-2014-03-26-Oculus Causes a Rift, but the Facebook Deal Will Avoid a Scaling Crisis for Virtual Reality
topicId topicWeight
[(0, 0.042), (1, 0.017), (2, 0.01), (3, -0.029), (4, 0.066), (5, -0.001), (6, -0.087), (7, 0.005), (8, 0.05), (9, 0.034), (10, 0.029), (11, 0.046), (12, 0.087), (13, 0.045), (14, -0.066), (15, 0.034), (16, 0.042), (17, -0.017), (18, 0.023), (19, 0.041), (20, 0.106), (21, 0.107), (22, 0.039), (23, 0.01), (24, 0.065), (25, 0.028), (26, 0.068), (27, -0.054), (28, 0.078), (29, -0.047), (30, -0.145), (31, 0.033), (32, 0.029), (33, 0.043), (34, -0.016), (35, 0.026), (36, 0.039), (37, -0.061), (38, -0.051), (39, 0.004), (40, -0.063), (41, -0.035), (42, 0.038), (43, 0.036), (44, -0.01), (45, -0.067), (46, -0.013), (47, 0.025), (48, 0.03), (49, -0.076)]
simIndex simValue blogId blogTitle
same-blog 1 0.99142891 562 high scalability-2009-04-10-Facebook's Aditya giving presentation on Facebook Architecture
Introduction: Facebook's engg. director aditya talks about facebook architecture. How they use mysql, php and memcache. How they have modified the above to suit their requirements.
2 0.83670938 845 high scalability-2010-06-22-Exploring the software behind Facebook, the world’s largest site
Introduction: Peter Alguacil at Pingdom wrote a HighScalability worthy article on Facebook's architecture: Exploring the software behind Facebook, the world’s largest site . It covers the challenges Facebook faces, the software Facebook uses, and the techniques Facebook uses to keep on scaling. Definitely worth a look.
3 0.79747045 264 high scalability-2008-03-03-Read This Site and Ace Your Next Interview!
Introduction: Paul Tyma published a massive and massively good 96 page insider's manual on How to Pass a Silicon Valley Software Engineering Interview . My eyes immediately latched on to one of his key example scenarios, which involves scaling Facebook: Facebook ● What was Facebook day 1? – A database with a PHP front-end ● In PHP, Java, C#, whatever – How long would it take you to reproduce Facebook's first incarnation? ● A single MySQL instance with some simple queries probably used to happily query the whole userbase. Facebook ● What is it today? ● Its not about “that stuff you learned in school” – Its about what a company with thousands of (possibly conflicting) queries per second operating on a directed-graph with 50 million nodes ● And of course a few Petabytes of data ● And 99.99% uptime ● Design decision? A Facebook user is (or recently was) currently limited to 5000 friends. If you've been reading all the wisdom contributed to and reference
4 0.75835377 624 high scalability-2009-06-10-Hive - A Petabyte Scale Data Warehouse using Hadoop
Introduction: This post about using Hive and Hadoop for analytics comes straight from Facebook engineers. Scalable analysis on large data sets has been core to the functions of a number of teams at Facebook - both engineering and non-engineering. Apart from ad hoc analysis and business intelligence applications used by analysts across the company, a number of Facebook products are also based on analytics. These products range from simple reporting applications like Insights for the Facebook Ad Network, to more advanced kind such as Facebook's Lexicon product. As a result a flexible infrastructure that caters to the needs of these diverse applications and users and that also scales up in a cost effective manner with the ever increasing amounts of data being generated on Facebook, is critical. Hive and Hadoop are the technologies that we have used to address these requirements at Facebook. Read the rest of the article on Engineering @ Facebook's Notes page
5 0.72191244 1323 high scalability-2012-09-15-4 Reasons Facebook Dumped HTML5 and Went Native
Introduction: Facebook made quite a splash when they released their native iOS app , not because of their app per se, but because of their conclusion that their biggest mistake was betting on HTML5 , so they had to go native. As you might imagine this was a bit like telling a Great White Shark that its bark is worse than its bite. A common refrain was Facebook simply had made a bad HTML5 site, not that HTML5 itself is bad, as plenty of other vendors have made slick well performing mobile sites. An interesting and relevant conversation given the rising butt kickery of mobile. But we were lacking details. Now we aren't. If you were wondering just why Facebook ditched HTML5, Tobie Langel in Perf Feedback - What's slowing down Mobile Facebook , lists out the reasons: Tooling / Developer APIs . Most importantly, the lack of tooling to track down memory problems. Scrolling performance. Scrolling must be fast and smooth and full featured. It's not. GPU. A clunky API and black box ap
6 0.69540268 720 high scalability-2009-10-12-High Performance at Massive Scale – Lessons learned at Facebook
7 0.67569059 599 high scalability-2009-05-14-Who Has the Most Web Servers?
8 0.66802204 966 high scalability-2010-12-31-Facebook in 20 Minutes: 2.7M Photos, 10.2M Comments, 4.6M Messages
9 0.66077721 1081 high scalability-2011-07-18-Building your own Facebook Realtime Analytics System
10 0.63000935 1619 high scalability-2014-03-26-Oculus Causes a Rift, but the Facebook Deal Will Avoid a Scaling Crisis for Virtual Reality
11 0.58621579 792 high scalability-2010-03-10-How FarmVille Scales - The Follow-up
13 0.55591232 1123 high scalability-2011-09-23-The Real News is Not that Facebook Serves Up 1 Trillion Pages a Month…
14 0.55259037 943 high scalability-2010-11-16-Facebook's New Real-time Messaging System: HBase to Store 135+ Billion Messages a Month
16 0.53308886 378 high scalability-2008-09-03-Some Facebook Secrets to Better Operations
17 0.52959991 563 high scalability-2009-04-10-Facebook Chat Architecture
18 0.52060372 870 high scalability-2010-08-02-7 Scaling Strategies Facebook Used to Grow to 500 Million Users
19 0.51337194 646 high scalability-2009-07-01-Podcast about Facebook's Cassandra Project and the New Wave of Distributed Databases
20 0.49309337 464 high scalability-2008-12-13-Strategy: Facebook Tweaks to Handle 6 Time as Many Memcached Requests
topicId topicWeight
[(28, 0.683), (79, 0.057)]
simIndex simValue blogId blogTitle
same-blog 1 0.73842728 562 high scalability-2009-04-10-Facebook's Aditya giving presentation on Facebook Architecture
Introduction: Facebook's engg. director aditya talks about facebook architecture. How they use mysql, php and memcache. How they have modified the above to suit their requirements.
2 0.46337056 630 high scalability-2009-06-14-kngine 'Knowledge Engine' milestone 2
Introduction: Kngine is Knowledge Web search engine designed to provide meaningful search results, such as: semantic information about the keywords/concepts, answer the user’s questions, discover the relations between the keywords/concepts, and link the different kind of data together, such as: Movies, Subtitles, Photos, Price at sale store, User reviews, and Influenced story Goals Kngine long-term goal is to make all human beings systematic knowledge and experience accessible to everyone. I aim to collect and organize all objective data, and make it possible and easy to access. Our goal is to build on the advances of Web search engine, semantic web, data representation technologies a new form of Web search engine that will unleash a revolution of new possibilities. Kngine tries to combine the power of Web search engines with the power of Semantic search and the data representation to provide meaningful search results compromising user needs. Status Kngine starts as a research project in O
3 0.37869334 1294 high scalability-2012-08-01-Prismatic Update: Machine Learning on Documents and Users
Introduction: In update to Prismatic Architecture - Using Machine Learning on Social Networks to Figure Out What You Should Read on the Web , Jason Wolfe, even in the face of deadening fatigue from long nights spent getting their iPhone app out, has gallantly agreed to talk a little more about Primatic's approach to Machine Learning. Documents and users are two areas where Prismatic applies ML (machine learning): ML on Documents Given an HTML document:Â learn how to extract the main text of the page (rather than the sidebar, footer, comments, etc), its title, author, best images, etc determine features for relevance (e.g., what the article is about, topics, etc.) The setup for most of these tasks is pretty typical. Models are trained using big batch jobs on other machines that read data from s3, save the learned parameter files to s3, and then read (and periodically refresh) the models from s3 in the ingest pipeline. All of the data that flows out of the system can be
4 0.3723444 606 high scalability-2009-05-25-non-sequential, unique identifier, strategy question
Introduction: (Please bare with me, I'm a new, passionate, confident and terrified programmer :D ) Background: I'm pre-launch and 1 year into the development of my application. My target is to be able to eventually handle millions of registered users with 5-10% of them concurrent. Up to this point I've used auto-increment to assign unique identifiers to rows. I am now considering switching to a non-sequential strategy. Oh, I'm using the LAMP configuration. My reasons for avoiding auto-increment: 1. Complicates replication when scaling horizontally. Risk of collision is significant (when running multiple masters). Note: I've read the other entries in this forum that relate to ID generation and there have been some great suggestions -- including a strategy that uses auto-increment in a way that avoids this pitfall... That said, I'm still nervous about it. 2. Potential bottleneck when retrieving/assigning IDs -- IDs assigned at the database. My reasons for being nervous about
5 0.24118346 752 high scalability-2009-12-17-Oracle and IBM databases: Disk-based vs In-memory databases
Introduction: Current disk based RDBMS can run out of steam when processing large data. Can these problems be solved by migrating from a disk based RDBMS to an IMDB? Any limitations? To find out, I tested one of each from the two leading vendors who together hold 70% of the market share - Oracle's 11g and TimesTen 11g , and IBM's DB2 v9.5 and solidDB 6.3 . read more at BigDataMatters.com
6 0.23594262 806 high scalability-2010-04-08-Hot Scalability Links for April 8, 2010
8 0.21271062 1506 high scalability-2013-08-23-Stuff The Internet Says On Scalability For August 23, 2013
9 0.21023399 903 high scalability-2010-09-17-Hot Scalability Links For Sep 17, 2010
10 0.18287165 1395 high scalability-2013-01-28-DuckDuckGo Architecture - 1 Million Deep Searches a Day and Growing
11 0.17539001 1611 high scalability-2014-03-12-Paper: Scalable Eventually Consistent Counters over Unreliable Networks
12 0.15703191 1261 high scalability-2012-06-08-Stuff The Internet Says On Scalability For June 8, 2012
13 0.11909264 1439 high scalability-2013-04-12-Stuff The Internet Says On Scalability For April 12, 2013
14 0.10948476 1201 high scalability-2012-02-29-Strategy: Put Mobile Video Into Cold Storage After 30 Days
15 0.09122736 692 high scalability-2009-09-01-Cheap storage: how backblaze takes matters in hand
16 0.09122736 1119 high scalability-2011-09-20-HighScalability is old news. Step your scaling game way up... (NSFW cartoon)
17 0.090772517 840 high scalability-2010-06-10-The Four Meta Secrets of Scaling at Facebook
18 0.090321071 8 high scalability-2007-07-12-Should I use LAMP or Windows?
19 0.090077981 743 high scalability-2009-11-23-Big Data on Grids or on Clouds?
20 0.088902906 782 high scalability-2010-02-23-When to migrate your database?