high_scalability high_scalability-2011 high_scalability-2011-1110 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: It's time to think of the architecture and application platforms surrounding "Big Data" databases. Big Data is often centered around new database technologies mostly from the emerging NoSQL world. The main challenge that these databases solve is how to handle massive amount of data at a reasonable cost and without poor performanc - distributed databases emerged to address this challenge and today we're seeing high adoption rate and quite impressive success stories such as the Netflix use of Cassandra/DataStax solution . All that indicate the speed in which this market evolves. The need for a Big Data Application Platform Application platforms provide a framework for making the development of applications simpler. They do this by carving out the generic parts of applications such as security, scalability, and reliability (which are attributes of a 'good' application) from the parts of the applications that are specific to our business domain. Most of the existing app
sentIndex sentText sentNum sentScore
1 It's time to think of the architecture and application platforms surrounding "Big Data" databases. [sent-1, score-0.448]
2 Big Data is often centered around new database technologies mostly from the emerging NoSQL world. [sent-2, score-0.098]
3 The need for a Big Data Application Platform Application platforms provide a framework for making the development of applications simpler. [sent-5, score-0.349]
4 They do this by carving out the generic parts of applications such as security, scalability, and reliability (which are attributes of a 'good' application) from the parts of the applications that are specific to our business domain. [sent-6, score-0.511]
5 Most of the existing application platforms such as Java EE and Ruby on Rails were designed to work with centralized relational databases in mind. [sent-7, score-0.436]
6 Clearly, that model doesn’t fit well to the Big Data world simply because it wasn’t designed to deal with massive amount of data in first place. [sent-8, score-0.354]
7 These include smart grid, marketing automation, clinical care, fraud detection and avoidance, criminal justice systems, cyber-security, and intelligence. [sent-10, score-0.465]
8 It requires a special class of developers to understand how to break their problem down into the components necessary for treatment by a distributed architecture like Hadoop. [sent-13, score-0.48]
9 For this model to take off, we need simpler models that are more accessible to a wider range of developers - while retaining all the power of these special platforms. [sent-14, score-0.644]
10 Other existing models for handling Big Data such as Data Warehouse don’t cut it either, as noted in Dan Woods ' post on Forbes, Big Data Requires a Big, New Architecture : . [sent-15, score-0.422]
11 to take maximum advantage of big data, IT is going to have to press the re-start button on its architecture for acquiring and understanding information. [sent-18, score-0.688]
12 IT will need to construct a new way of capturing, organizing and analyzing data, because big data stands no chance of being useful if people attempt to process it using the traditional mechanisms of business intelligence, such as a data warehouses and traditional data-analysis techniques. [sent-19, score-1.31]
13 Here's my personal view on how that platform could look like based on my experience covering the NoSQL space for a while now and through my experience with GigaSpaces. [sent-21, score-0.079]
wordName wordTfidf (topN-words)
[('big', 0.34), ('noted', 0.186), ('data', 0.176), ('platforms', 0.155), ('models', 0.155), ('clinical', 0.131), ('justice', 0.131), ('woods', 0.123), ('forrester', 0.117), ('carving', 0.113), ('retaining', 0.109), ('forbes', 0.109), ('treatment', 0.109), ('criminal', 0.109), ('applications', 0.109), ('application', 0.108), ('special', 0.107), ('hotter', 0.106), ('pioneers', 0.104), ('avoidance', 0.101), ('ee', 0.101), ('warehouses', 0.101), ('emerged', 0.101), ('challenge', 0.098), ('centered', 0.098), ('developers', 0.097), ('surrounding', 0.096), ('fraud', 0.094), ('databases', 0.092), ('capturing', 0.092), ('parts', 0.09), ('acquiring', 0.09), ('massive', 0.09), ('traditional', 0.09), ('architecture', 0.089), ('organizing', 0.089), ('model', 0.088), ('wider', 0.088), ('hadoop', 0.088), ('press', 0.087), ('stands', 0.087), ('framework', 0.085), ('construct', 0.084), ('button', 0.082), ('indicate', 0.081), ('existing', 0.081), ('space', 0.079), ('attributes', 0.079), ('requires', 0.078), ('attempt', 0.077)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999994 1110 high scalability-2011-09-06-Big Data Application Platform
Introduction: It's time to think of the architecture and application platforms surrounding "Big Data" databases. Big Data is often centered around new database technologies mostly from the emerging NoSQL world. The main challenge that these databases solve is how to handle massive amount of data at a reasonable cost and without poor performanc - distributed databases emerged to address this challenge and today we're seeing high adoption rate and quite impressive success stories such as the Netflix use of Cassandra/DataStax solution . All that indicate the speed in which this market evolves. The need for a Big Data Application Platform Application platforms provide a framework for making the development of applications simpler. They do this by carving out the generic parts of applications such as security, scalability, and reliability (which are attributes of a 'good' application) from the parts of the applications that are specific to our business domain. Most of the existing app
2 0.21751107 1216 high scalability-2012-03-27-Big Data In the Cloud Using Cloudify
Introduction: Edd Dumbill wrote an interesting article on O’Reilly Radar covering the current solutions for running Big Data in the Cloud Big data and cloud technology go hand-in-hand. Big data needs clusters of servers for processing, which clouds can readily provide. Big PaaS Edd touched briefly on the role of PaaS for delivering Big Data applications in the cloud Beyond IaaS, several cloud services provide application layer support for big data work. Sometimes referred to as managed solutions, or platform as a service (PaaS), these services remove the need to ucale things such as databases or MapReduce, reducing your workload and maintenance burden. Additionally, PaaS providers can realize great efficiencies by hosting at the application level, and pass those savings on to the customer. To put it simply, managing data clusters is one thing. Being able to process the data is yet another challenge that we need to think about when we’re dealing with application platforms, as I no
3 0.19669856 954 high scalability-2010-12-06-What the heck are you actually using NoSQL for?
Introduction: It's a truism that we should choose the right tool for the job . Everyone says that. And who can disagree? The problem is this is not helpful advice without being able to answer more specific questions like: What jobs are the tools good at? Will they work on jobs like mine? Is it worth the risk to try something new when all my people know something else and we have a deadline to meet? How can I make all the tools work together? In the NoSQL space this kind of real-world data is still a bit vague. When asked, vendors tend to give very general answers like NoSQL is good for BigData or key-value access. What does that mean for for the developer in the trenches faced with the task of solving a specific problem and there are a dozen confusing choices and no obvious winner? Not a lot. It's often hard to take that next step and imagine how their specific problems could be solved in a way that's worth taking the trouble and risk. Let's change that. What problems are you using NoSQL to sol
Introduction: All in all this is still my favorite post and I still think it's an accurate vision of a future. Not everyone agrees, but I guess we'll see..."But it is not complicated. [There's] just a lot of it." \--Richard Feynmanon how the immense variety of the world arises from simple rules.Contents:Have We Reached the End of Scaling?Applications Become Black Boxes Using Markets to Scale and Control CostsLet's Welcome our Neo-Feudal OverlordsThe Economic Argument for the Ambient CloudWhat Will Kill the Cloud?The Amazing Collective Compute Power of the Ambient CloudUsing the Ambient Cloud as an Application RuntimeApplications as Virtual StatesConclusionWe have not yet begun to scale. The world is still fundamentally disconnected and for all our wisdom we are still in the earliest days of learning how to build truly large planet-scaling applications.Today 350 million users on Facebook is a lot of users and five million followers on Twitter is a lot of followers. This may seem like a lot now, but c
Introduction: "But it is not complicated. [There's] just a lot of it." \--Richard Feynmanon how the immense variety of the world arises from simple rules.Contents:Have We Reached the End of Scaling?Applications Become Black Boxes Using Markets to Scale and Control CostsLet's Welcome our Neo-Feudal OverlordsThe Economic Argument for the Ambient CloudWhat Will Kill the Cloud?The Amazing Collective Compute Power of the Ambient CloudUsing the Ambient Cloud as an Application RuntimeApplications as Virtual StatesConclusionWe have not yet begun to scale. The world is still fundamentally disconnected and for all our wisdom we are still in the earliest days of learning how to build truly large planet-scaling applications.Today 350 million users on Facebook is a lot of users and five million followers on Twitter is a lot of followers. This may seem like a lot now, but consider we have no planet wide applications yet. None.Tomorrow the numbers foreshadow a newCambrian explosionof connectivity that will look as
6 0.14987358 538 high scalability-2009-03-16-Are Cloud Based Memory Architectures the Next Big Thing?
7 0.1461231 1240 high scalability-2012-05-07-Startups are Creating a New System of the World for IT
8 0.14492771 450 high scalability-2008-11-24-Scalability Perspectives #3: Marc Andreessen – Internet Platforms
9 0.13991836 931 high scalability-2010-10-28-Notes from A NOSQL Evening in Palo Alto
10 0.1365553 1064 high scalability-2011-06-20-35+ Use Cases for Choosing Your Next NoSQL Database
11 0.13460037 1313 high scalability-2012-08-28-Making Hadoop Run Faster
13 0.13186792 1514 high scalability-2013-09-09-Need Help with Database Scalability? Understand I-O
14 0.1318645 1056 high scalability-2011-06-09-Retrospect on recent AWS outage and Resilient Cloud-Based Architecture
topicId topicWeight
[(0, 0.251), (1, 0.055), (2, 0.066), (3, 0.09), (4, 0.038), (5, 0.062), (6, -0.034), (7, -0.049), (8, 0.027), (9, 0.065), (10, 0.008), (11, 0.084), (12, -0.017), (13, 0.007), (14, 0.042), (15, -0.047), (16, 0.034), (17, -0.064), (18, 0.006), (19, 0.016), (20, -0.039), (21, 0.039), (22, 0.113), (23, -0.024), (24, 0.074), (25, 0.001), (26, -0.009), (27, -0.068), (28, -0.027), (29, 0.036), (30, -0.001), (31, 0.057), (32, -0.0), (33, 0.018), (34, -0.06), (35, 0.045), (36, -0.034), (37, -0.016), (38, 0.046), (39, 0.003), (40, 0.014), (41, -0.033), (42, -0.022), (43, 0.03), (44, 0.061), (45, -0.057), (46, 0.025), (47, -0.033), (48, -0.069), (49, 0.026)]
simIndex simValue blogId blogTitle
same-blog 1 0.96513963 1110 high scalability-2011-09-06-Big Data Application Platform
Introduction: It's time to think of the architecture and application platforms surrounding "Big Data" databases. Big Data is often centered around new database technologies mostly from the emerging NoSQL world. The main challenge that these databases solve is how to handle massive amount of data at a reasonable cost and without poor performanc - distributed databases emerged to address this challenge and today we're seeing high adoption rate and quite impressive success stories such as the Netflix use of Cassandra/DataStax solution . All that indicate the speed in which this market evolves. The need for a Big Data Application Platform Application platforms provide a framework for making the development of applications simpler. They do this by carving out the generic parts of applications such as security, scalability, and reliability (which are attributes of a 'good' application) from the parts of the applications that are specific to our business domain. Most of the existing app
2 0.80519199 1216 high scalability-2012-03-27-Big Data In the Cloud Using Cloudify
Introduction: Edd Dumbill wrote an interesting article on O’Reilly Radar covering the current solutions for running Big Data in the Cloud Big data and cloud technology go hand-in-hand. Big data needs clusters of servers for processing, which clouds can readily provide. Big PaaS Edd touched briefly on the role of PaaS for delivering Big Data applications in the cloud Beyond IaaS, several cloud services provide application layer support for big data work. Sometimes referred to as managed solutions, or platform as a service (PaaS), these services remove the need to ucale things such as databases or MapReduce, reducing your workload and maintenance burden. Additionally, PaaS providers can realize great efficiencies by hosting at the application level, and pass those savings on to the customer. To put it simply, managing data clusters is one thing. Being able to process the data is yet another challenge that we need to think about when we’re dealing with application platforms, as I no
3 0.79306394 1161 high scalability-2011-12-22-Architecting Massively-Scalable Near-Real-Time Risk Analysis Solutions
Introduction: Constructing a scalable risk analysis solution is a fascinating architectural challenge. If you come from Financial Services you are sure to appreciate that. But even architects from other domains are bound to find the challenges fascinating, and the architectural patterns of my suggested solution highly useful in other domains. Recently I held an interesting webinar around architecting solutions for scalable and near-real-time risk analysis solutions based on experience gathered with Financial Services customers. Seeing the vast interest in the webinar, I would like to share the highlights with you here. From an architectural point of view, risk analysis is a data-intensive and a compute-intensive process, which also has an elaborate orchestration logic. volumes in this domain are massive and ever-increasing, together with an ever-increasing demand to reduce response time. These trends are aggravated by global financial regulatory reforms set following the late-2000s
Introduction: This is a guest post by Eric Czech , Chief Architect at Next Big Sound, talks about some unique approaches taken to solving scalability challenges in music analytics. Tracking online activity is hardly a new idea, but doing it for the entire music industry isn't easy. Half a billion music video streams, track downloads, and artist page likes occur each day and measuring all of this activity across platforms such as Spotify, iTunes, YouTube, Facebook, and more, poses some interesting scalability challenges. Next Big Sound collects this type of data from over a hundred sources, standardizes everything, and offers that information to record labels, band managers, and artists through a web-based analytics platform. While many of our applications use open-source systems like Hadoop, HBase, Cassandra, Mongo, RabbitMQ, and MySQL, our usage is fairly standard, but there is one aspect of what we do that is pretty unique. We collect or receive information from 100+ sources and we s
5 0.73895526 1292 high scalability-2012-07-27-Stuff The Internet Says On Scalability For July 27, 2012
Introduction: It's HighScalability Time: Almost 1 Billion Users: Facebook ; 30,000 connections across 94 locations: the Olympics Network ; 2.5 quintillion: bytes of data created each day ; 80K QPS: MemSQL . In some early results Zencoder found EC2 was faster than GCE in their video transcoding tests, saying Google needs larger instances with faster CPUs. Love how Google's jbeda said they would take a look at the results. +1 for competition and benchmarks. Something to keep in mind is for Google a core means : a hyperthread per virtual CPU. So that means that a n1-standard-8 instance gets 4 physical cores and 8 hyperthreads, not 8 physical cores. Kevin Rose recommends hiring generalists rather than developers with niche skills; don't give away your company; and thinks advisors should be investors. Founders should also probably stick around and managers shouldn't blame developers. Is MemSQL the world's fastest database? BS meter on high, but it is created by two former Facebook
6 0.73535419 727 high scalability-2009-10-25-Is Your Data Really Secured?
7 0.7345311 885 high scalability-2010-08-23-Building a Scalable Key-Value Database: Project Hydracus
8 0.73364127 697 high scalability-2009-09-09-GridwiseTech revolutionizes data management
9 0.73058039 250 high scalability-2008-02-17-Web Accelerators - snake oil or miracle remedy?
10 0.72954661 1092 high scalability-2011-08-04-Jim Starkey is Creating a Brave New World by Rethinking Databases for the Cloud
11 0.72884917 1430 high scalability-2013-03-27-The Changing Face of Scale - The Downside of Scaling in the Contextual Age
12 0.72827065 954 high scalability-2010-12-06-What the heck are you actually using NoSQL for?
13 0.7274437 809 high scalability-2010-04-13-Strategy: Saving Your Butt With Deferred Deletes
14 0.72643471 1087 high scalability-2011-07-26-Web 2.0 Killed the Middleware Star
15 0.72485238 822 high scalability-2010-05-04-Business continuity with real-time data integration
16 0.72013581 1015 high scalability-2011-04-01-Stuff The Internet Says On Scalability For April 1, 2011
17 0.71700138 744 high scalability-2009-11-24-Hot Scalability Links for Nov 24 2009
18 0.7139746 351 high scalability-2008-07-16-The Mother of All Database Normalization Debates on Coding Horror
19 0.71217048 126 high scalability-2007-10-20-Should you build your next website using 3tera's grid OS?
20 0.71169817 1160 high scalability-2011-12-21-In Memory Data Grid Technologies
topicId topicWeight
[(1, 0.217), (2, 0.191), (10, 0.046), (30, 0.014), (49, 0.029), (61, 0.09), (79, 0.127), (91, 0.141), (94, 0.069)]
simIndex simValue blogId blogTitle
1 0.97416252 722 high scalability-2009-10-15-Hot Scalability Links for Oct 15 2009
Introduction: Update: Social networks in the database: using a graph database . Anders Nawroth puts graphs through their paces by representing, traversing, and performing other common social network operations using a graph database. Update: Deployment with Capistrano by Charles Max Wood. Simple step-by-step for using Capistrano for deployment. Log-structured file systems: There's one in every SSD by Valerie Aurora. SSDs have totally changed the performance characteristics of storage! Disks are dead! Long live flash! An Engineer's Guide to Bandwidth by DGentry. I t's a rough world out there, and we need to to a better job of thinking about and testing under realistic network conditions. Analyzing air traffic performance with InfoBright and MonetDB by Vadim of the MySQL Performance Blog. Scalable Delivery of Stream Query Result by Zhou, Y ; Salehi, A ; Aberer, K. In this paper, we leverage Distributed Publish/Subscribe System (DPSS), a scalable data dissemination infrastruct
2 0.94653332 742 high scalability-2009-11-17-10 eBay Secrets for Planet Wide Scaling
Introduction: You don't even have to make a bid, Randy Shoup, an eBay Distinguished Architect, gives this presentation on how eBay scales, for free. Randy has done a fabulous job in this presentation and in other talks listed at the end of this post getting at the heart of the principles behind scalability. It's more about ideas of how things work and fit together than a focusing on a particular technology stack. Impressive Stats In case you weren't sure, eBay is big, with lots of: users, data, features, and change... Over 89 million active users worldwide 190 million items for sale in 50,000 categories Over 8 billion URL requests per day Hundreds of new features per quarter Roughly 10% of items are listed or ended every day In 39 countries and 10 languages 24x7x365 70 billion read / write operations / day Processes 50TB of new, incremental data per day Analyzes 50PB of data per day 10 Lessons The presentation does a good job explaining each lesson, but the list is.
same-blog 3 0.94471711 1110 high scalability-2011-09-06-Big Data Application Platform
Introduction: It's time to think of the architecture and application platforms surrounding "Big Data" databases. Big Data is often centered around new database technologies mostly from the emerging NoSQL world. The main challenge that these databases solve is how to handle massive amount of data at a reasonable cost and without poor performanc - distributed databases emerged to address this challenge and today we're seeing high adoption rate and quite impressive success stories such as the Netflix use of Cassandra/DataStax solution . All that indicate the speed in which this market evolves. The need for a Big Data Application Platform Application platforms provide a framework for making the development of applications simpler. They do this by carving out the generic parts of applications such as security, scalability, and reliability (which are attributes of a 'good' application) from the parts of the applications that are specific to our business domain. Most of the existing app
4 0.94097424 453 high scalability-2008-12-01-Breakthrough Web-Tier Solutions with Record-Breaking Performance
Introduction: With the explosive growth of the Internet, increasing complexity of user requirements, and wide choice of hardware, operating systems, and middleware, IT executives are facing new challenges in their application infrastructures. Rapid expansion of the application tier has resulted in significant cost and complexity, and many organizations are simply running out of datacenter space, power, and cooling.
5 0.9333697 1093 high scalability-2011-08-05-Stuff The Internet Says On Scalability For August 5, 2011
Introduction: Submitted for your beginning of the end of summer scaling pleasure: Google Uses About 900,000 Servers ; eBay deploys 100TB of flash storage The cloud isn't for closers. Another gaming startup pulls back from the cloud by Derrick Harris. Digital Chocolate is following the Zynga strategy of moving games into higher performing datacenter infrastructure once it becomes popular enough in the cloud to justify the primo stuff. We talked about this strategy in Zynga's Z Cloud - Scale Fast Or Fail Fast By Merging Private And Public Clouds . An architectural approach made all the more sensible with Amazon's new AWS Direct Connect service, which enables lower latency and higher bandwidth services by skipping the Internet and connecting directly to the AWS network. AWS Direct Connect FAQs . Amazon Virtual Private Cloud . Quotes that are quotable: @Werner : "If You Are Slow, You Can't Grow" - Peecho Architecture - scalability on a shoestring http://wv.ly/n4fpPC #aws
6 0.92965412 356 high scalability-2008-07-22-Scaling Bumper Sticker: A 1 Billion Page Per Month Facebook RoR App
7 0.91812563 712 high scalability-2009-10-01-Moving Beyond End-to-End Path Information to Optimize CDN Performance
8 0.91460305 167 high scalability-2007-11-27-Starting a website from scratch - what technologies should I use?
9 0.91412926 1053 high scalability-2011-06-06-Apple iCloud: Syncing and Distributed Storage Over Streaming and Centralized Storage
10 0.90983903 1216 high scalability-2012-03-27-Big Data In the Cloud Using Cloudify
11 0.90939087 840 high scalability-2010-06-10-The Four Meta Secrets of Scaling at Facebook
12 0.90899581 1557 high scalability-2013-12-02-Evolution of Bazaarvoice’s Architecture to 500M Unique Users Per Month
13 0.90873963 576 high scalability-2009-04-21-What CDN would you recommend?
14 0.90808862 1338 high scalability-2012-10-11-RAMCube: Exploiting Network Proximity for RAM-Based Key-Value Store
15 0.90781504 841 high scalability-2010-06-14-How scalable could be a cPanel Hosting service?
16 0.90625 826 high scalability-2010-05-12-The Rise of the Virtual Cellular Machines
17 0.90562332 1514 high scalability-2013-09-09-Need Help with Database Scalability? Understand I-O
18 0.90441072 853 high scalability-2010-07-08-Cloud AWS Infrastructure vs. Physical Infrastructure
19 0.90379852 888 high scalability-2010-08-27-OpenStack - The Answer to: How do We Compete with Amazon?
20 0.90257049 126 high scalability-2007-10-20-Should you build your next website using 3tera's grid OS?