high_scalability high_scalability-2011 knowledge-graph by maker-knowledge-mining

high_scalability 2011 knowledge graph

similar blogs computed by tfidf model

similar blogs computed by lsi model

similar blogs computed by lda model

blogs list:

1 high scalability-2011-12-30-Stuff The Internet Says On Scalability For December 30, 2011

Introduction: Pork. The Other HighScalability: PlentyOfFish: 6 Billion Page Views ; World: info doubling every 2 years ; 2015: 7,910 exabytes of global digital data ; Khan Academy: 4 million uniques ; G+: 62 million users ; Zynga: leased 9 megawatts of capacity ; Heroku: billions of page views a month Quoteable quotes: Udi Dhan : Scalability is not boolean. John Boyd: Look at the mission, not the technology. And if you do look at the mission, don't look at the most fashionable mission of the day. @BigDataClouds : I think the fear of change is the biggest challenge that companies are facing @cjzero : If you were wondering, the #Mythbusters scalability test of a Newton's Cradle using wrecking balls? Busted. @Xorlev : Scalability is really really hard. That's why it's fun. It pushes the limits of engineering talent. 100 Best Cloud & Data Stats of 2011 by Zenoss. Lots of fun facts about how mind bendingly huge the world of information is exponentially be

2 high scalability-2011-12-28-Strategy: Guaranteed Availability Requires Reserving Instances in Specific Zones

Introduction: When EC2 first started the mental model was of a magic Pez dispenser supplying an infinite stream of instances in any desired flavor. If you needed an instance, because of a either a failure or traffic spike, it would be there. As amazing as EC2 is, this model turned out to be optimistic. From a thread on the Amazon discussion forum we learn any dispenser has limits: As Availability Zones grow over time, our ability to continue to expand them can become constrained. In these scenarios, we will prevent customers from launching in the constrained zone if they do not yet have existing resources in that zone. We also might remove the constrained zone entirely from the list of options for new customers. This means that occasionally, different customers will see a different number of Availability Zones in a particular Region. Both approaches aim to help customers avoid accidentally starting to build up their infrastructure in an Availability Zone where they might have less ability

3 high scalability-2011-12-27-PlentyOfFish Update - 6 Billion Pageviews and 32 Billion Images a Month

Introduction: Markus has a short update on their PlentyOfFish Architecture . Impressive November statistics: 6 billion pageviews served 32 billion images served 6 million logins i n one day IM servers handle about 30 billion pageviews 11 webservers (5 of which could be dropped) Hired first DBA in July . They currently have a handful of employees . All hosting/cdn costs combined are under $70k/month. Lesson : small organization, simple architecture, on raw hardware is still plenty profitable for PlentyOfFish. Related Articles On HackerNews 32 Billion images a month by Markus Frind.

4 high scalability-2011-12-23-Stuff The Internet Says On Scalability For December 23, 2011

Introduction: A merry HighScalability to all and to all a good night: Santa: 3.7 million appointments; iPad2 == 1986 Cray 2 6 processor super computer ; Watson: 200 million pages of natural language content Funny: a cautionary tale about storage and backup . Where is my data? I’m kinda big deal after all! I should have listened to my postdoc, he can build cheaper storage than you can. Nothing stirs up more energy than when someone says they are abandoning and old beloved framework for a newer sexier model. Feelings of betrayal and abandonment leak over everything, which is always a good draw for reality social networking. Here Paul Querna tells why The Switch: Python to Node.js . And here we see the response on Hacker News . Python got the job done for cloudkick, they were acquired, but they wanted something more going forward, a trophy wife if you will, after the first wife put them through law school. Good discussion all around. You may find something that helps in your own platfo

5 high scalability-2011-12-23-Funny: A Cautionary Tale About Storage and Backup

Introduction: Storage: everyone wants it, but nobody wants to pay for it. This is one of those funny cartoons showing the doom of that sort of thinking...pay me now or cry over lost data later. Loved: Where is my data? I’m kinda big deal after all! I should have listened to my postdoc, he can build cheaper storage than you can. I should have went to Newegg. Found via Joe Landman and James Cuff . Joe talks about how RAID isn't backup and has some more wit and wisdom.

6 high scalability-2011-12-22-Architecting Massively-Scalable Near-Real-Time Risk Analysis Solutions

Introduction: Constructing a scalable risk analysis solution is a fascinating architectural challenge. If you come from Financial Services you are sure to appreciate that. But even architects from other domains are bound to find the challenges fascinating, and the architectural patterns of my suggested solution highly useful in other domains. Recently I held an interesting webinar around architecting solutions for scalable and near-real-time risk analysis solutions based on experience gathered with Financial Services customers. Seeing the vast interest in the webinar, I would like to share the highlights with you here. From an architectural point of view, risk analysis is a data-intensive and a compute-intensive process, which also has an elaborate orchestration logic. volumes in this domain are massive and ever-increasing, together with an ever-increasing demand to reduce response time. These trends are aggravated by global financial regulatory reforms set following the late-2000s

7 high scalability-2011-12-21-In Memory Data Grid Technologies

Introduction: After winning a CSC Leading Edge Forum (LEF) research grant, I (Paul Colmer) wanted to publish some of the highlights of my research to share with the wider technology community. What is an In Memory Data Grid? It is not an in-memory relational database, a NOSQL database or a relational database. It is a different breed of software datastore. In summary an IMDG is an ‘off the shelf’ software product that exhibits the following characteristics: The data model is distributed across many servers in a single location or across multiple locations. This distribution is known as a data fabric. This distributed model is known as a ‘shared nothing’ architecture. All servers can be active in each site. All data is stored in the RAM of the servers. Servers can be added or removed non-disruptively, to increase the amount of RAM available. The data model is non-relational and is object-based. Distributed applications written on the .NET and Java application platforms are s

8 high scalability-2011-12-19-How Twitter Stores 250 Million Tweets a Day Using MySQL

Introduction: Jeremy Cole , a DBA Team Lead/Database Architect at Twitter, gave a really good talk at the O'Reilly MySQL conference: Big and Small Data at @Twitter , where the topic was thinking of Twitter from the data perspective. One of the interesting stories he told was of the transition from Twitter's old way of storing tweets using temporal sharding , to a more distributed approach using a new tweet store called T-bird, which is built on top of Gizzard , which is built using MySQL. Twitter's original tweet store: Temporally sharded tweets was a good-idea-at-the-time architecture. Temporal sharding simply means tweets from the same date range are stored together on the same shard. The problem is tweets filled up one machine, then a second, and then a third. You end up filling up one machine after another. This is a pretty common approach and one that has some real flaws: Load balancing . Most of the old machines didn't get any traffic because people

9 high scalability-2011-12-16-Stuff The Internet Says On Scalability For December 16, 2011

Introduction: A HighScalability is forever: eBay: tens of millions of lines of code; Google code base change rate per month: 50% ; Apple: 100 million downloads ; Internet: 186 Gbps Quotable quotes: @OttmarAmann : Scalability is not as important as managing complexity @amankapur91 : Does scalability imply standardization, and then does standardization imply loss of innovation? Spotify uses a P2P architecture and this paper, Spotify – Large Scale, Low Latency, P2P Music-on-Demand Streaming , describes it. The Faving spam counter-measures . Ironically, deviantART relates a gripping story of how they detected and stopped a deviant user from attacking their servers with an automated faving script which faved every 10 seconds for 24 hours a day. The same spam filter they use on the rest of the site was used. Problem solved. Would like detail on their spam filter though. Interesting Google Group's thread on the best practices for simulating transactions

10 high scalability-2011-12-14-Virtualization and Cloud Computing is Changing the Network to East-West Routing

Introduction: It’s called “east-west” networking, which when compared to its predecessor, “north-south” networking, evinces images of maelstroms and hurricane winds and tsunamis for some reason. It could be the subtle correlation between the transformative shift this change in networking patterns has on the data center with that of El Niño’s transformative power upon the weather patterns across the globe. Traditionally, data center networks have focused on North-South network traffic. The assumption is that clients on the edge would mainly communicate with servers at the core, rather than across the network to other clients. But server virtualization changes all this, with servers, virtual appliances and even virtual desktops scattered across the same physical infrastructure. These environments are also highly dynamic, with workloads moving to different physical locations on the network as virtual servers are migrated (in the case of data center networks) and clients move

11 high scalability-2011-12-13-Sponsored Post: Cedexis, Callfire, Attribution Modeling, Logic Monitor, New Relic, ScaleOut, AppDynamics, CloudSigma, ManageEngine, Site24x7

Introduction: Who's Hiring? Callfire, one of the largest cloud telephony platforms on the web, is hiring a Sr. Software Engineer. You can learn more here . Fun and Informative Events Sign up for this free 30-minute webinar exploring how new technology can determine which ads have been seen by users and will discuss the C3 Metrics Labs analysis of over 2 billion impressions. Cool Products and Services Not satisfied with performance in the cloud? Visit Cedexis and Lose the Wait . Looking for a path to the Hybrid Cloud? Cedexis can help you find the right path . LogicMonitor - Hosted monitoring of your entire technology stack. Dashboards, trending graphs, alerting. Try it free and be up and running in just 15 minutes. New Relic - real user monitoring optimize for humans, not bots. Live application stats, SQL/NoSQL performance, web transactions, proactive notifications. Take 2 minutes to sign up for a free trial. ScaleOut StateServer® Delivers Map/Reduce An

12 high scalability-2011-12-12-Netflix: Developing, Deploying, and Supporting Software According to the Way of the Cloud

Introduction: At a Cloud Computing Meetup , Siddharth "Sid" Anand of Netflix, backed by a merry band of Netflixians, gave an interesting talk: Keeping Movies Running Amid Thunderstorms . While the talk gave a good overview of their move to the cloud, issues with capacity planning, thundering herds , latency problems, and simian armageddon , I found myself most taken with how they handle software deployment in the cloud . I've worked on half a dozen or more build and deployment systems, some small, some quite large, but never for a large organization like Netflix in the cloud. The cloud has this amazing capability that has never existed before that enables a novel approach to fault-tolerant software deployments: the ability to spin up huge numbers of instances to completely run a new release while running the old release at the same time . The process goes something like: A canary machine is launched first with the new software load running real traffic to sanity test the load in a p

13 high scalability-2011-12-09-Stuff The Internet Says On Scalability For December 9, 2011

Introduction: It takes a licking and keeps on HighScalabilitying: Instagram: 60 photos per second ; Twitter: 8,868 Tweets per second MTV Awards ; Foursquare: 15 million users ; Facebook: 60 million qps ; Netflix: stream 1 billion hours in Q4 ; Evolution: 16,000 eyes ; Evernote: 750K Paid Users ; AOL: 60,000 Servers ; Netflix API: 20,000 requests per second ; Tumblr: 900 posts per second . Quotable quotes: @LusciousPear : This is my key-value store. There are many others like it, but this one is mine. @pavlobaron : @old_sound if this site isn't on highscalability.com , I won't open it @nzmrmn : Words I hate: moist, "sinfully decadent," facilitator, dopesauce, deliverables, accountability, coolio, "ping me," scalability. @anggriawan : Mark Callaghan (Facebook Engineer): "You can tune #MySQL to perform very fast per node if you know what you’re doing." Jesper Nordenberg : Java is good at hiding the real complexities of programm

14 high scalability-2011-12-08-Update on Scalable Causal Consistency For Wide-Area Storage With COPS

Introduction: Here are a few updates on the article Paper: Don’t Settle For Eventual: Scalable Causal Consistency For Wide-Area Storage With COPS from Mike Freedman and Wyatt Lloyd. Q: How software architectures could change in response to casual+ consistency? A : I don't really think they would much. Somebody would still run a two-tier architecture in their datacenter: a front-tier of webservers running both (say) PHP and our client library, and a back tier of storage nodes running COPS. (I'm not sure if it was obvious given the discussion of our "thick" client -- you should think of the COPS client dropping in where a memcache client library does...albeit ours has per-session state.) Q: Why not just use vector clocks? A : The problem with vector clocks and scalability has always been that the size of vector clocks in O(N), where N is the number of nodes. So if we want to scale to a datacenter with 10K nodes, each piece of metadata must have size O(10K). And in fact, vector

15 high scalability-2011-12-06-Instagram Architecture: 14 Million users, Terabytes of Photos, 100s of Instances, Dozens of Technologies

Introduction: Instagram is a free photo sharing and social networking service for your iPhone that has been an instant success . Growing to 14 million users in just over a year, they reached 150 million photos in August while amassing several terabytes of photos, and they did this with just 3 Instaneers, all on the Amazon stack. The Instagram team has written up what can be considered the canonical description of an early stage startup in this era: What Powers Instagram: Hundreds of Instances, Dozens of Technologies . Instagram uses a pastiche of different technologies and strategies. The team is small yet has experience rapid growth riding the crest of a rising social and mobile wave, it uses a hybrid of SQL and NoSQL, it uses a ton of open source projects, they chose the cloud over colo, Amazon services are highly leveraged rather than building their own, reliability is through availability zones, async work scheduling links components together, the system is composed as much as possible

16 high scalability-2011-12-05-Stuff The Internet Says On Scalability For December 5, 2011

Introduction: It's HighScalability Time! Quotable quotes: @jaykreps : Was wondering, How can I turn my boring, cachable, read-only traffic into random writes on mongodb? And lo! link @marshallk : Google runs 100-200 experiments every day on UI, algorithm & product @styggiti : The problem with companies like IBM and Oracle baking NoSQL "scalability" into their products isn't the tech, it's the $$ licensing. Blazing fast node.js: 10 performance tips from LinkedIn Mobile . You may have thought that node.js made just everything magically fast, but Shravya Garlapati has some great strategies for going even faster: Avoid synchronous code; Turn off socket pooling; Don't use Node.js for static assets; Render on the client-side; Use gzip; Go parallel; Go session-free; Use binary modules; Use standard V8 JavaScript instead of client-side libraries; Keep your code small and light. Nice thread in NoSQL Databases on HBase and Consistency in CAP . The short summary of the a

17 high scalability-2011-12-02-Stuff The Internet Says On Scalability For December 2, 2011

Introduction: Sorry, this edition of Stuff the Internet on Scalability has been called on the account of two straight days of whipping, whirling, wind that has left me powerless to complete the post. Service will resume when Mother Nature is nicer. Pardon me while I try and find our roof...

18 high scalability-2011-11-29-Sponsored Post: Cedexis, Callfire, Attribution Modeling, Logic Monitor, New Relic, ScaleOut, Percona Live MySQL, AppDynamics, CloudSigma, ManageEngine, Site24x7

Introduction: Who's Hiring? Callfire, one of the largest cloud telephony platforms on the web, is hiring a Sr. Software Engineer. You can learn more here . Fun and Informative Events Sign up for this free 30-minute webinar exploring how new technology can determine which ads have been seen by users and will discuss the C3 Metrics Labs analysis of over 2 billion impressions. Come one come all! Introducing Percona Live: MySQL Conference And Expo 2012 . Join us for this three day intensive MySQL conference April 10th-12th, 2012. Cool Products and Services Not satisfied with performance in the cloud? Visit Cedexis and Lose the Wait . Looking for a path to the Hybrid Cloud? Cedexis can help you find the right path . LogicMonitor - Hosted monitoring of your entire technology stack. Dashboards, trending graphs, alerting. Be up and running in 15 minutes. New Relic - real user monitoring optimize for humans, not bots. Live application stats, SQL/NoSQL performance,

19 high scalability-2011-11-29-DataSift Architecture: Realtime Datamining at 120,000 Tweets Per Second

Introduction: I remember the excitement of when Twitter first opened up their firehose. As an early adopter of the Twitter API I could easily imagine some of the cool things you could do with all that data. I also remember the disappointment of learning that in the land of BigData, data has a price, and that price would be too high for little fish like me. It was like learning for the first time there would be no BigData Santa Clause. For a while though I had the pleasure of pondering just how I would handle all that data. It's a fascinating problem. You have to be able to reliably consume it, normalize it, merge it with other data, apply functions on it, store it, query it, distribute it, and oh yah, monetize it. Most of that in realish-time. And if you are trying to create a platform for allowing the entire Internet do to the same thing to the firehose, the challenge is exponentially harder. DataSift is in the exciting position of creating just such a firehose eating, data chomping machine. Y

20 high scalability-2011-11-25-Stuff The Internet Says On Scalability For November 25, 2011

Introduction: A HighScalability a day keeps the fail whale away.: 46 million turkeys eaten at Thanksgiving ; Pinterest 421 million pageviews Quotable quotes: @al3xandr3 : scalability demands decoupling @startupandrew : "Yesterday I did some digestive system scalability testing" -- in reference to eating 140 chicken nuggets Peter Wayner : While the crazy dreamers can continue to craft NoSQL data stores, serious people will want to take a look at Oracle's version. @weblearning : Unitil we get scalability - or that wonderful word massification - in education, we will not get value for money Embrace and extend: @conor_omahony : IBM is Baking NoSQL Capabilities into DB2 and Informix. @GGopman : I look for scalability. That's what turns me on Supercomputing: An Industry in Need of a Revolution . Bartosz Milewski wants decently paying, interesting and meaningful jobs for all (supercomputer programmers). Get on that Santa. Caching a

21 high scalability-2011-11-23-Paper: Don’t Settle for Eventual: Scalable Causal Consistency for Wide-Area Storage with COPS

22 high scalability-2011-11-18-Stuff The Internet Says On Scalability For November 18, 2011

23 high scalability-2011-11-17-Five Misconceptions on Cloud Portability

24 high scalability-2011-11-16-Google+ Infrastructure Update - the JavaScript Story

25 high scalability-2011-11-14-Using Gossip Protocols for Failure Detection, Monitoring, Messaging and Other Good Things

26 high scalability-2011-11-11-Stuff The Internet Says On Scalability For November 11, 2011

27 high scalability-2011-11-10-Kill the Telcos Save the Internet - The Unsocial Network

28 high scalability-2011-11-08-Sponsored Post: New Relic, Atlassian, ScaleOut, aiCache, Rocketfuel, FreeAgent, Percona Live!, AppDynamics, Couchbase, CloudSigma, ManageEngine, Site24x7

29 high scalability-2011-11-07-10 Core Architecture Pattern Variations for Achieving Scalability

30 high scalability-2011-11-04-Stuff The Internet Says On Scalability For November 4, 2011

31 high scalability-2011-11-03-Paper: G2 : A Graph Processing System for Diagnosing Distributed Systems

32 high scalability-2011-10-31-15 Ways to Make Your Application Feel More Responsive under Google App Engine