high_scalability high_scalability-2008 high_scalability-2008-309 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: The recent Data-Intensive Computing Symposium brought together experts in system design, programming, parallel algorithms, data management, scientific applications, and information-based applications to better understand existing capabilities in the development and application of large-scale computing systems, and to explore future opportunities. Google Fellow Jeff Dean had a very interesting presentation on Handling Large Datasets at Google: Current Systems and Future Directions. He discussed: • Hardware infrastructure • Distributed systems infrastructure: –Scheduling system –GFS –BigTable –MapReduce • Challenges and Future Directions –Infrastructure that spans all datacenters –More automation It is really like a "How does Google work" presentation in ~60 slides? Check out the slides and the video !
sentIndex sentText sentNum sentScore
1 Google Fellow Jeff Dean had a very interesting presentation on Handling Large Datasets at Google: Current Systems and Future Directions. [sent-2, score-0.305]
wordName wordTfidf (topN-words)
[('thevideo', 0.27), ('symposium', 0.252), ('presentation', 0.247), ('future', 0.245), ('fellow', 0.224), ('gfs', 0.211), ('directions', 0.211), ('spans', 0.211), ('dean', 0.191), ('scientific', 0.191), ('infrastructure', 0.185), ('discussed', 0.169), ('slides', 0.167), ('bigtable', 0.158), ('jeff', 0.152), ('scheduling', 0.146), ('datasets', 0.144), ('brought', 0.144), ('computing', 0.142), ('explore', 0.139), ('experts', 0.137), ('systems', 0.128), ('datacenters', 0.128), ('google', 0.127), ('capabilities', 0.121), ('recent', 0.117), ('mapreduce', 0.111), ('check', 0.105), ('algorithms', 0.103), ('challenges', 0.102), ('applications', 0.1), ('handling', 0.1), ('understand', 0.097), ('existing', 0.093), ('current', 0.092), ('parallel', 0.085), ('together', 0.08), ('programming', 0.08), ('development', 0.07), ('management', 0.069), ('hardware', 0.065), ('system', 0.061), ('design', 0.06), ('interesting', 0.058), ('better', 0.053), ('really', 0.049), ('large', 0.046), ('distributed', 0.046), ('application', 0.041), ('work', 0.041)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999988 309 high scalability-2008-04-23-Behind The Scenes of Google Scalability
Introduction: The recent Data-Intensive Computing Symposium brought together experts in system design, programming, parallel algorithms, data management, scientific applications, and information-based applications to better understand existing capabilities in the development and application of large-scale computing systems, and to explore future opportunities. Google Fellow Jeff Dean had a very interesting presentation on Handling Large Datasets at Google: Current Systems and Future Directions. He discussed: • Hardware infrastructure • Distributed systems infrastructure: –Scheduling system –GFS –BigTable –MapReduce • Challenges and Future Directions –Infrastructure that spans all datacenters –More automation It is really like a "How does Google work" presentation in ~60 slides? Check out the slides and the video !
2 0.19006076 448 high scalability-2008-11-22-Google Architecture
Introduction: Update 2: Sorting 1 PB with MapReduce . PB is not peanut-butter-and-jelly misspelled. It's 1 petabyte or 1000 terabytes or 1,000,000 gigabytes. It took six hours and two minutes to sort 1PB (10 trillion 100-byte records) on 4,000 computers and the results were replicated thrice on 48,000 disks. Update: Greg Linden points to a new Google article MapReduce: simplified data processing on large clusters . Some interesting stats: 100k MapReduce jobs are executed each day; more than 20 petabytes of data are processed per day; more than 10k MapReduce programs have been implemented; machines are dual processor with gigabit ethernet and 4-8 GB of memory. Google is the King of scalability. Everyone knows Google for their large, sophisticated, and fast searching, but they don't just shine in search. Their platform approach to building scalable applications allows them to roll out internet scale applications at an alarmingly high competition crushing rate. Their goal is always to build
3 0.17235148 734 high scalability-2009-10-30-Hot Scalabilty Links for October 30 2009
Introduction: Life beyond Distributed Transactions: an Apostate’s Opinion by Pat Helland. In particular, we focus on the implications that fall out of assuming we cannot have large-scale distributed transactions. T ragedy of the Commons, and Cold Starts - Cold application starts on Google App Engine kill your application's responsiveness. Intel’s 1M IOPS desktop SSD setup by Kevin Burton. What do you get when you take 7 Intel SSDs and throw them in a desktop? 1M IOPS Videos from NoSQL Berlin sessions. Nicely done talks on CAP, MongoDB, Redis, 4th generation object databases, CouchDB, and Riak. Designs, Lessons and Advice from Building Large Distributed Systems by Jeff Dean of Google describing how they do their thing. Here are some glosses on the talk by Greg Linden and James Hamilton. You really can't do better than Greg and James. Advice from Google on Large Distributed Systems by Greg Linden. A nice summary of Jeff Dean's talk. A standard Google server
4 0.17094447 652 high scalability-2009-07-08-Art of Parallelism presentation
Introduction: This presentation about parallel computing, and it’s discover the following topic: What is parallelism? Why now? How it’s works? What is the current options Parallel Runtime Library. (for more information go there ) Note: All of my presentation is open source, so feel free to copy it, use it, and re-distribute it. Download
5 0.15133707 409 high scalability-2008-10-13-Challenges from large scale computing at Google
Introduction: From Greg Linden on a talk Google Fellow Jeff Dean gave last week at University of Washington Computer Science titled "Research Challenges Inspired by Large-Scale Computing at Google" : Coming away from the talk, the biggest points for me were the considerable interest in reducing costs (especially reducing power costs), the suggestion that the Google cluster may eventually contain 10M machines at 1k locations, and the call to action for researchers on distributed systems and databases to think orders of magnitude bigger than they often are, not about running on hundreds of machines in one location, but hundreds of thousands of machines across many locations.
6 0.15116072 650 high scalability-2009-07-02-Product: Hbase
7 0.13580884 1535 high scalability-2013-10-21-Google's Sanjay Ghemawat on What Made Google Google and Great Big Data Career Advice
8 0.1206507 601 high scalability-2009-05-17-Product: Hadoop
9 0.11841603 818 high scalability-2010-04-30-Behind the scenes of an online marketplace
10 0.11533232 693 high scalability-2009-09-03-Storage Systems for High Scalable Systems presentation
11 0.11178408 1328 high scalability-2012-09-24-Google Spanner's Most Surprising Revelation: NoSQL is Out and NewSQL is In
12 0.10937231 590 high scalability-2009-05-06-Art of Distributed
13 0.10281848 517 high scalability-2009-02-21-Google AppEngine - A Second Look
14 0.10172834 483 high scalability-2009-01-04-Paper: MapReduce: Simplified Data Processing on Large Clusters
15 0.098053783 644 high scalability-2009-06-29-eHarmony.com describes how they use Amazon EC2 and MapReduce
17 0.096221671 750 high scalability-2009-12-16-Building Super Scalable Systems: Blade Runner Meets Autonomic Computing in the Ambient Cloud
19 0.094786867 223 high scalability-2008-01-25-Google: Introduction to Distributed System Design
20 0.091896832 1049 high scalability-2011-05-31-Awesome List of Advanced Distributed Systems Papers
topicId topicWeight
[(0, 0.135), (1, 0.031), (2, 0.059), (3, 0.122), (4, -0.031), (5, 0.029), (6, -0.001), (7, -0.005), (8, 0.007), (9, 0.123), (10, 0.008), (11, -0.03), (12, -0.007), (13, -0.013), (14, 0.026), (15, -0.035), (16, -0.061), (17, -0.053), (18, 0.087), (19, 0.018), (20, 0.074), (21, 0.064), (22, -0.013), (23, -0.059), (24, 0.021), (25, 0.02), (26, 0.062), (27, 0.072), (28, -0.09), (29, 0.027), (30, 0.042), (31, 0.003), (32, 0.032), (33, 0.029), (34, -0.021), (35, -0.037), (36, 0.067), (37, 0.015), (38, 0.06), (39, 0.075), (40, 0.011), (41, 0.013), (42, -0.008), (43, 0.012), (44, -0.04), (45, -0.127), (46, -0.018), (47, -0.076), (48, 0.044), (49, 0.067)]
simIndex simValue blogId blogTitle
same-blog 1 0.9797892 309 high scalability-2008-04-23-Behind The Scenes of Google Scalability
Introduction: The recent Data-Intensive Computing Symposium brought together experts in system design, programming, parallel algorithms, data management, scientific applications, and information-based applications to better understand existing capabilities in the development and application of large-scale computing systems, and to explore future opportunities. Google Fellow Jeff Dean had a very interesting presentation on Handling Large Datasets at Google: Current Systems and Future Directions. He discussed: • Hardware infrastructure • Distributed systems infrastructure: –Scheduling system –GFS –BigTable –MapReduce • Challenges and Future Directions –Infrastructure that spans all datacenters –More automation It is really like a "How does Google work" presentation in ~60 slides? Check out the slides and the video !
2 0.82799709 362 high scalability-2008-08-11-Distributed Computing & Google Infrastructure
Introduction: A couple of videos about distributed computing with direct reference on Google infrastructure. You will get acquainted with: --MapReduce the software framework implemented by Google to support parallel computations over large (greater than 100 terabyte) data sets on commodity hardware --GFS and the way it stores it's data into 64mb chunks --Bigtable which is the simple implementation of a non-relational database at Google Cluster Computing and MapReduce Lectures 1-5 .
3 0.81396872 409 high scalability-2008-10-13-Challenges from large scale computing at Google
Introduction: From Greg Linden on a talk Google Fellow Jeff Dean gave last week at University of Washington Computer Science titled "Research Challenges Inspired by Large-Scale Computing at Google" : Coming away from the talk, the biggest points for me were the considerable interest in reducing costs (especially reducing power costs), the suggestion that the Google cluster may eventually contain 10M machines at 1k locations, and the call to action for researchers on distributed systems and databases to think orders of magnitude bigger than they often are, not about running on hundreds of machines in one location, but hundreds of thousands of machines across many locations.
4 0.74186796 734 high scalability-2009-10-30-Hot Scalabilty Links for October 30 2009
Introduction: Life beyond Distributed Transactions: an Apostate’s Opinion by Pat Helland. In particular, we focus on the implications that fall out of assuming we cannot have large-scale distributed transactions. T ragedy of the Commons, and Cold Starts - Cold application starts on Google App Engine kill your application's responsiveness. Intel’s 1M IOPS desktop SSD setup by Kevin Burton. What do you get when you take 7 Intel SSDs and throw them in a desktop? 1M IOPS Videos from NoSQL Berlin sessions. Nicely done talks on CAP, MongoDB, Redis, 4th generation object databases, CouchDB, and Riak. Designs, Lessons and Advice from Building Large Distributed Systems by Jeff Dean of Google describing how they do their thing. Here are some glosses on the talk by Greg Linden and James Hamilton. You really can't do better than Greg and James. Advice from Google on Large Distributed Systems by Greg Linden. A nice summary of Jeff Dean's talk. A standard Google server
5 0.72245198 223 high scalability-2008-01-25-Google: Introduction to Distributed System Design
Introduction: Update: Google added videos on Cluster Computing and MapReduce . There are five lectures: Introduction, MapReduce, Distributed File Systems, Clustering Algorithms, and Graph Algorithms . Advanced website design depends on deep distributed system design knowledge. Where do you get this knowledge? Try Google. They have a a whole Code for Educators program with tutorials and lectures on AJAX programming, distributed systems, and web security. Looks pretty nice.
6 0.71685737 1535 high scalability-2013-10-21-Google's Sanjay Ghemawat on What Made Google Google and Great Big Data Career Advice
7 0.70656598 483 high scalability-2009-01-04-Paper: MapReduce: Simplified Data Processing on Large Clusters
8 0.69903773 1328 high scalability-2012-09-24-Google Spanner's Most Surprising Revelation: NoSQL is Out and NewSQL is In
10 0.68372065 871 high scalability-2010-08-04-Dremel: Interactive Analysis of Web-Scale Datasets - Data as a Programming Paradigm
11 0.66691875 650 high scalability-2009-07-02-Product: Hbase
13 0.65183598 1107 high scalability-2011-08-29-The Three Ages of Google - Batch, Warehouse, Instant
14 0.64206809 652 high scalability-2009-07-08-Art of Parallelism presentation
15 0.6412015 284 high scalability-2008-03-19-RAD Lab is Creating a Datacenter Operating System
16 0.6385808 211 high scalability-2008-01-13-Google Reveals New MapReduce Stats
17 0.63779032 401 high scalability-2008-10-04-Is MapReduce going mainstream?
18 0.62942517 590 high scalability-2009-05-06-Art of Distributed
19 0.62794608 1540 high scalability-2013-10-30-Strategy: Use Your Quantum Computer Lab to Tell Intentional Blinks from Involuntary Blinks
20 0.60926354 640 high scalability-2009-06-28-Google Voice Architecture
topicId topicWeight
[(1, 0.177), (2, 0.213), (4, 0.211), (10, 0.034), (79, 0.238)]
simIndex simValue blogId blogTitle
1 0.9370963 1343 high scalability-2012-10-18-Save up to 30% by Selecting Better Performing Amazon Instances
Introduction: If you like the idea of exploiting market inconsistencies to lower your costs then you will love this paper and video from the Hot Cloud '12 conference: Exploiting Hardware Heterogeneity within the Same Instance Type of Amazon EC2 . The conclusion is interesting and is a source of good guidance: Amazon EC2 uses diversified hardware to host the same type of instance. The hardware diversity results in performance variation. In general, the variation between the fast instances and slow instances can reach 40%. In some applications, the variation can even approach up to 60%. By selecting fast instances within the same instance type, Amazon EC2 users can acquire up to 30% of cost saving, if the fast instances have a relatively low probability. The abstract: Cloud computing providers might start with near-homogeneous hardware environment. Over time, the homogeneous environment will most likely evolve into heterogeneous one because of possible upgrades and replac
same-blog 2 0.92145491 309 high scalability-2008-04-23-Behind The Scenes of Google Scalability
Introduction: The recent Data-Intensive Computing Symposium brought together experts in system design, programming, parallel algorithms, data management, scientific applications, and information-based applications to better understand existing capabilities in the development and application of large-scale computing systems, and to explore future opportunities. Google Fellow Jeff Dean had a very interesting presentation on Handling Large Datasets at Google: Current Systems and Future Directions. He discussed: • Hardware infrastructure • Distributed systems infrastructure: –Scheduling system –GFS –BigTable –MapReduce • Challenges and Future Directions –Infrastructure that spans all datacenters –More automation It is really like a "How does Google work" presentation in ~60 slides? Check out the slides and the video !
3 0.88153094 79 high scalability-2007-09-01-On-Demand Infinitely Scalable Database Seed the Amazon EC2 Cloud
Introduction: Amazon's EC2 sounds good, but how do you make use of all that throbbing CPU power? A few companies are stepping up to fill the how-to gap. Elastra provides unlimited on-demand creation of MySQL and PostgresSQL instances for $.50/server/hour. They contend their clusters perform "nearly" as well as a local database deployed using local storage. RightScale says they "enable you to run your entire web business on Amazon Web Services with reliability, scalability and performance – and pushbutton control of complex system administration tasks." This includes web servers, DNS, and MySQL services. Prices start at $500 a month. Later I'll write more about these and other related services like 3tera , but these services are the canary in the coal mine, the face of change, the bellwether of the new data center. How we build scalable web sites is about to change.
4 0.87564063 12 high scalability-2007-07-15-Isilon Clustred Storage System
Introduction: The Isilon IQ family of clustered storage systems was designed from the ground up to meet the needs of data-intensive enterprises and high-performance computing environments. By combining Isilon's OneFS® operating system software with the latest advances in industry-standard hardware, Isilon delivers modular, pay-as-you-grow, enterprise-class clustered storage systems. OneFS, with TrueScale™ technology, powers the industry's first and only storage system that enables linear or independent scaling of performance and capacity. This new flexible and tunable system, featuring a robust suite of clustered storage software applications, provides customers with an "out of the box" solution that is fully optimized for the widest range of applications and workflow needs. * Scales from 4 TB ti 1 PB * Throughput of up to 10 GB per seond * Linear scaling * Easy to manage Related Articles Inside Skinny On Isilon by StorageMojo
Introduction: It’s called “east-west” networking, which when compared to its predecessor, “north-south” networking, evinces images of maelstroms and hurricane winds and tsunamis for some reason. It could be the subtle correlation between the transformative shift this change in networking patterns has on the data center with that of El Niño’s transformative power upon the weather patterns across the globe. Traditionally, data center networks have focused on North-South network traffic. The assumption is that clients on the edge would mainly communicate with servers at the core, rather than across the network to other clients. But server virtualization changes all this, with servers, virtual appliances and even virtual desktops scattered across the same physical infrastructure. These environments are also highly dynamic, with workloads moving to different physical locations on the network as virtual servers are migrated (in the case of data center networks) and clients move
7 0.84982741 448 high scalability-2008-11-22-Google Architecture
9 0.84707499 919 high scalability-2010-10-14-I, Cloud
10 0.84370244 1213 high scalability-2012-03-22-Paper: Revisiting Network I-O APIs: The netmap Framework
11 0.84366608 680 high scalability-2009-08-13-Reconnoiter - Large-Scale Trending and Fault-Detection
12 0.84360576 380 high scalability-2008-09-05-Product: Tungsten Replicator
13 0.84340429 282 high scalability-2008-03-18-Database War Stories #3: Flickr
14 0.84265363 38 high scalability-2007-07-30-Build an Infinitely Scalable Infrastructure for $100 Using Amazon Services
15 0.84032208 786 high scalability-2010-03-02-Using the Ambient Cloud as an Application Runtime
16 0.84028345 1328 high scalability-2012-09-24-Google Spanner's Most Surprising Revelation: NoSQL is Out and NewSQL is In
17 0.83953387 1485 high scalability-2013-07-01-PRISM: The Amazingly Low Cost of Using BigData to Know More About You in Under a Minute
18 0.83794606 1240 high scalability-2012-05-07-Startups are Creating a New System of the World for IT
19 0.83682346 1589 high scalability-2014-02-03-How Google Backs Up the Internet Along With Exabytes of Other Data
20 0.83399832 1494 high scalability-2013-07-19-Stuff The Internet Says On Scalability For July 19, 2013