high_scalability high_scalability-2008 high_scalability-2008-294 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Hi, I am building a video-sharing site and I'm looking for an efficient way to update video views count. The easiest way would be to perform an SQL update to increase the "views" counter every time a video is viewed, but naturally I want to avoid DB write access as much as possible. I am looking for an efficient temporary storage to which I could connect and say "increment views of video X". Every so often I would save the changes to my main database, and remove the counter from this temporary storage. I am having a hard time finding such temporary storage, however. My first thought was memcache, but it's not ideal as I wouldn't like to lose the data if memcache goes down. Also, memcache's increment command requires that the key is already present - that means that every time a video is viewed, I would have to check if the key already exists in memcache, before I can actually send the increment command. What do people use to solve this kind of issues? Kind regar
sentIndex sentText sentNum sentScore
1 Hi, I am building a video-sharing site and I'm looking for an efficient way to update video views count. [sent-1, score-0.887]
2 The easiest way would be to perform an SQL update to increase the "views" counter every time a video is viewed, but naturally I want to avoid DB write access as much as possible. [sent-2, score-1.423]
3 I am looking for an efficient temporary storage to which I could connect and say "increment views of video X". [sent-3, score-1.305]
4 Every so often I would save the changes to my main database, and remove the counter from this temporary storage. [sent-4, score-1.017]
5 I am having a hard time finding such temporary storage, however. [sent-5, score-0.57]
6 My first thought was memcache, but it's not ideal as I wouldn't like to lose the data if memcache goes down. [sent-6, score-0.669]
7 Also, memcache's increment command requires that the key is already present - that means that every time a video is viewed, I would have to check if the key already exists in memcache, before I can actually send the increment command. [sent-7, score-2.396]
8 What do people use to solve this kind of issues? [sent-8, score-0.181]
wordName wordTfidf (topN-words)
[('increment', 0.46), ('temporary', 0.377), ('memcache', 0.349), ('viewed', 0.257), ('video', 0.255), ('views', 0.241), ('counter', 0.215), ('efficient', 0.129), ('easiest', 0.127), ('kind', 0.12), ('naturally', 0.117), ('exists', 0.112), ('update', 0.107), ('ideal', 0.106), ('would', 0.104), ('already', 0.102), ('every', 0.098), ('looking', 0.095), ('command', 0.095), ('lose', 0.092), ('connect', 0.09), ('finding', 0.087), ('remove', 0.086), ('key', 0.083), ('present', 0.081), ('db', 0.08), ('check', 0.07), ('avoid', 0.07), ('send', 0.068), ('main', 0.068), ('storage', 0.066), ('perform', 0.065), ('save', 0.064), ('thought', 0.063), ('time', 0.061), ('solve', 0.061), ('requires', 0.06), ('way', 0.06), ('goes', 0.059), ('sql', 0.058), ('increase', 0.056), ('actually', 0.056), ('changes', 0.053), ('say', 0.052), ('issues', 0.051), ('often', 0.05), ('means', 0.046), ('hard', 0.045), ('write', 0.045), ('access', 0.043)]
simIndex simValue blogId blogTitle
same-blog 1 1.0 294 high scalability-2008-04-01-How to update video views count effectively?
Introduction: Hi, I am building a video-sharing site and I'm looking for an efficient way to update video views count. The easiest way would be to perform an SQL update to increase the "views" counter every time a video is viewed, but naturally I want to avoid DB write access as much as possible. I am looking for an efficient temporary storage to which I could connect and say "increment views of video X". Every so often I would save the changes to my main database, and remove the counter from this temporary storage. I am having a hard time finding such temporary storage, however. My first thought was memcache, but it's not ideal as I wouldn't like to lose the data if memcache goes down. Also, memcache's increment command requires that the key is already present - that means that every time a video is viewed, I would have to check if the key already exists in memcache, before I can actually send the increment command. What do people use to solve this kind of issues? Kind regar
2 0.20195958 1201 high scalability-2012-02-29-Strategy: Put Mobile Video Into Cold Storage After 30 Days
Introduction: Limelight says 95% of Mobile Video Views Take Place in First 90 Days and 88.8 percent of views take place in the first 30 days. Since a lot of people are working with video, which is expensive to store and serve, the implication: there's little need to keep your video close to the user or on a CDN after 30 days.
Introduction: While working with Memcache the other night, it dawned on me that it’s usage as a distributed caching mechanism was really just one of many ways to use it. That there are in fact many alternative usages that one could find for Memcache if they could just realize what Memcache really is at its core – a simple distributed hash-table – is an important point worthy of further discussion. To be clear, when I say “simple”, by no means am I implying that Memcache’s implementation is simple, just that the ideas behind it are such. Think about that for a minute. What else could we use a simple distributed hash-table for, besides caching? How about using it as an alternative to the traditional shard lookup method we used in our Master Index Lookup scalability strategy, discussed previously here.
4 0.15584077 796 high scalability-2010-03-16-Justin.tv's Live Video Broadcasting Architecture
Introduction: The future is live. The future is real-time. The future is now. That's the hype anyway. And as it has a habit of doing, the hype is slowly becoming reality. We are seeing live searches, live tweets, live location, live reality augmentation, live crab (fresh and local), and live event publishing. One of the most challenging of all live technologies is that of live video broadcasting. Imagine a world in which everyone becomes a broadcaster and a consumer of video streams, all in real-time (< 250 msec latency), all so you can talk and interact directly without feeling like you are in the middle of a time shift war. The resources and the engineering needed to make this happened must be substantial. How do you do that? To find out I talked to Kyle Vogt, Justin.tv Founder and VP of Engineering. Justin.tv certainly has the numbers. Their 30 million unique monthly visitors even outshine YouTube in the video upload game, reportedly uploading nearly 30 hours per minute of video compared to Y
5 0.15490529 1359 high scalability-2012-11-15-Gone Fishin': Justin.Tv's Live Video Broadcasting Architecture
Introduction: This is one of my favorite posts for a couple of reasons. I think it gives a lot of useful information in an interesting space. And Kyle Vogt was just a real pleasure to talk to. He was very helpful and forthcoming, which makes the whole experience better for everyone. The future is live. The future is real-time. The future is now. That's the hype anyway. And as it has a habit of doing, the hype is slowly becoming reality. We are seeing live searches, live tweets, live location, live reality augmentation, live crab (fresh and local), and live event publishing. One of the most challenging of all live technologies is that of live video broadcasting. Imagine a world in which everyone becomes a broadcaster and a consumer of video streams, all in real-time (< 250 msec latency), all so you can talk and interact directly without feeling like you are in the middle of a time shift war. The resources and the engineering needed to make this happened must be substantial. How do you do tha
6 0.15201373 274 high scalability-2008-03-12-YouTube Architecture
7 0.15137973 991 high scalability-2011-02-16-Paper: An Experimental Investigation of the Akamai Adaptive Video Streaming
9 0.1323455 927 high scalability-2010-10-26-Marrying memcached and NoSQL
10 0.1247203 1215 high scalability-2012-03-26-7 Years of YouTube Scalability Lessons in 30 Minutes
11 0.11651234 1037 high scalability-2011-05-10-Viddler Architecture - 7 Million Embeds a Day and 1500 Req-Sec Peak
12 0.11549658 577 high scalability-2009-04-22-Gear6 Web cache - the hardware solution for working with Memcache
13 0.11083484 1126 high scalability-2011-09-27-Use Instance Caches to Save Money: Latency == $$$
14 0.10709035 313 high scalability-2008-05-02-Friends for Sale Architecture - A 300 Million Page View-Month Facebook RoR App
15 0.10204345 1440 high scalability-2013-04-15-Scaling Pinterest - From 0 to 10s of Billions of Page Views a Month in Two Years
16 0.10011063 52 high scalability-2007-08-01-Product: Memcached
17 0.097259581 514 high scalability-2009-02-18-Numbers Everyone Should Know
18 0.096589789 239 high scalability-2008-02-04-Streaming Video on Amazon EC2?
19 0.095972329 828 high scalability-2010-05-17-7 Lessons Learned While Building Reddit to 270 Million Page Views a Month
20 0.092684969 1220 high scalability-2012-04-02-YouPorn - Targeting 200 Million Views a Day and Beyond
topicId topicWeight
[(0, 0.119), (1, 0.062), (2, -0.039), (3, -0.057), (4, 0.022), (5, -0.023), (6, -0.029), (7, 0.01), (8, 0.023), (9, -0.01), (10, -0.007), (11, -0.032), (12, -0.061), (13, 0.019), (14, 0.032), (15, 0.058), (16, -0.03), (17, 0.024), (18, -0.026), (19, -0.087), (20, -0.043), (21, 0.001), (22, 0.056), (23, 0.043), (24, -0.003), (25, 0.004), (26, 0.005), (27, 0.029), (28, 0.042), (29, 0.04), (30, 0.012), (31, -0.025), (32, -0.017), (33, 0.019), (34, -0.003), (35, -0.006), (36, -0.022), (37, -0.085), (38, 0.075), (39, 0.057), (40, 0.025), (41, 0.04), (42, -0.037), (43, -0.008), (44, 0.076), (45, -0.079), (46, 0.005), (47, 0.037), (48, 0.096), (49, 0.012)]
simIndex simValue blogId blogTitle
same-blog 1 0.96216387 294 high scalability-2008-04-01-How to update video views count effectively?
Introduction: Hi, I am building a video-sharing site and I'm looking for an efficient way to update video views count. The easiest way would be to perform an SQL update to increase the "views" counter every time a video is viewed, but naturally I want to avoid DB write access as much as possible. I am looking for an efficient temporary storage to which I could connect and say "increment views of video X". Every so often I would save the changes to my main database, and remove the counter from this temporary storage. I am having a hard time finding such temporary storage, however. My first thought was memcache, but it's not ideal as I wouldn't like to lose the data if memcache goes down. Also, memcache's increment command requires that the key is already present - that means that every time a video is viewed, I would have to check if the key already exists in memcache, before I can actually send the increment command. What do people use to solve this kind of issues? Kind regar
2 0.83285093 1201 high scalability-2012-02-29-Strategy: Put Mobile Video Into Cold Storage After 30 Days
Introduction: Limelight says 95% of Mobile Video Views Take Place in First 90 Days and 88.8 percent of views take place in the first 30 days. Since a lot of people are working with video, which is expensive to store and serve, the implication: there's little need to keep your video close to the user or on a CDN after 30 days.
3 0.77040011 991 high scalability-2011-02-16-Paper: An Experimental Investigation of the Akamai Adaptive Video Streaming
Introduction: Video is hot on the Internet and people are really interested in knowing how to make it work. Dan Rayburn has a post pointing to a fascinating paper: An Experimental Investigation of the Akamai Adaptive Video Streaming , which talks in some detail about the protocols big players like YouTube, Skype and Akamai use to serve video over on an inherently video unfriendly medium like the Internet. For Akamai they found: Each video is encoded in five versions at different bit rates and stored in separate files. The client sends commands to the server with an average inter departure time of about 2 s, i.e. the control algorithm is executed on average each 2 seconds. Akamai uses only the video level to adapt the video source to the available bandwidth, whereas the frame rate of the video is kept constant. When a sudden drop in the available bandwidth occurs, short interruptions of the video playback can occur due to the a large actuation delay. For a sudden increase of the avai
4 0.75584686 796 high scalability-2010-03-16-Justin.tv's Live Video Broadcasting Architecture
Introduction: The future is live. The future is real-time. The future is now. That's the hype anyway. And as it has a habit of doing, the hype is slowly becoming reality. We are seeing live searches, live tweets, live location, live reality augmentation, live crab (fresh and local), and live event publishing. One of the most challenging of all live technologies is that of live video broadcasting. Imagine a world in which everyone becomes a broadcaster and a consumer of video streams, all in real-time (< 250 msec latency), all so you can talk and interact directly without feeling like you are in the middle of a time shift war. The resources and the engineering needed to make this happened must be substantial. How do you do that? To find out I talked to Kyle Vogt, Justin.tv Founder and VP of Engineering. Justin.tv certainly has the numbers. Their 30 million unique monthly visitors even outshine YouTube in the video upload game, reportedly uploading nearly 30 hours per minute of video compared to Y
5 0.75556177 1359 high scalability-2012-11-15-Gone Fishin': Justin.Tv's Live Video Broadcasting Architecture
Introduction: This is one of my favorite posts for a couple of reasons. I think it gives a lot of useful information in an interesting space. And Kyle Vogt was just a real pleasure to talk to. He was very helpful and forthcoming, which makes the whole experience better for everyone. The future is live. The future is real-time. The future is now. That's the hype anyway. And as it has a habit of doing, the hype is slowly becoming reality. We are seeing live searches, live tweets, live location, live reality augmentation, live crab (fresh and local), and live event publishing. One of the most challenging of all live technologies is that of live video broadcasting. Imagine a world in which everyone becomes a broadcaster and a consumer of video streams, all in real-time (< 250 msec latency), all so you can talk and interact directly without feeling like you are in the middle of a time shift war. The resources and the engineering needed to make this happened must be substantial. How do you do tha
6 0.7279563 274 high scalability-2008-03-12-YouTube Architecture
7 0.71247709 198 high scalability-2008-01-01-HOW CDN works
8 0.70890224 1215 high scalability-2012-03-26-7 Years of YouTube Scalability Lessons in 30 Minutes
9 0.69275999 1037 high scalability-2011-05-10-Viddler Architecture - 7 Million Embeds a Day and 1500 Req-Sec Peak
10 0.68514168 239 high scalability-2008-02-04-Streaming Video on Amazon EC2?
11 0.67934275 447 high scalability-2008-11-19-High Definition Video Delivery on the Web?
12 0.63229632 123 high scalability-2007-10-15-Olympic Site Architecture
13 0.61918539 576 high scalability-2009-04-21-What CDN would you recommend?
14 0.58034319 39 high scalability-2007-07-30-Product: Akamai
15 0.57269758 238 high scalability-2008-02-04-IPS-IDS for heavy content site
16 0.56922758 60 high scalability-2007-08-07-Can you profit from the coming Content Delivery Network wars?
17 0.55336213 181 high scalability-2007-12-11-Hosting and CDN for startup video sharing site
18 0.54410255 828 high scalability-2010-05-17-7 Lessons Learned While Building Reddit to 270 Million Page Views a Month
19 0.53943366 1493 high scalability-2013-07-17-Steve Ballmer Says Microsoft has Over 1 Million Servers - What Does that Really Mean?
20 0.53553277 1289 high scalability-2012-07-23-State of the CDN: More Traffic, Stable Prices, More Products, Profits - Not So Much
topicId topicWeight
[(1, 0.128), (2, 0.243), (30, 0.015), (31, 0.125), (61, 0.112), (79, 0.15), (85, 0.091)]
simIndex simValue blogId blogTitle
1 0.96772492 1651 high scalability-2014-05-20-It's Networking. In Space! Or How E.T. Will Phone Home.
Introduction: What will the version of the Internet that follows us to the stars look like? Yes, people are really thinking seriously about this sort of thing. Specifically the InterPlanetary Networking Special Interest Group (IPNSIG). Ansible-like faster-than-light communication it isn't. There's no magical warp drive. Nor is a network of telepaths acting as a 'verse spanning telegraph system. It's more mundane than that. And in many ways more interesting as it's sort of like the old Internet on steroids, the one that was based on on UUCP and dial-up connections, but over vastly longer distances and with much longer delays : The Interplanetary Internet (based on IPN, also called InterPlaNet) is a conceived computer network in space, consisting of a set of network nodes which can communicate with each other.[1][2] Communication would be greatly delayed by the great interplanetary distances, so the IPN needs a new set of protocols and technology that are tolerant to large delays and
same-blog 2 0.9661476 294 high scalability-2008-04-01-How to update video views count effectively?
Introduction: Hi, I am building a video-sharing site and I'm looking for an efficient way to update video views count. The easiest way would be to perform an SQL update to increase the "views" counter every time a video is viewed, but naturally I want to avoid DB write access as much as possible. I am looking for an efficient temporary storage to which I could connect and say "increment views of video X". Every so often I would save the changes to my main database, and remove the counter from this temporary storage. I am having a hard time finding such temporary storage, however. My first thought was memcache, but it's not ideal as I wouldn't like to lose the data if memcache goes down. Also, memcache's increment command requires that the key is already present - that means that every time a video is viewed, I would have to check if the key already exists in memcache, before I can actually send the increment command. What do people use to solve this kind of issues? Kind regar
3 0.95021719 785 high scalability-2010-02-26-MySQL and Memcached: End of an Era?
Introduction: If you look at the early days of this blog, when web scalability was still in its heady bloom of youth, many of the articles had to do with leveraging MySQL and memcached . Exciting times. Shard MySQL to handle high write loads, cache objects in memcached to handle high read loads, and then write a lot of glue code to make it all work together. That was state of the art, that was how it was done. The architecture of many major sites still follow this pattern today, largely because with enough elbow grease, it works. This was a pre-cloud, relational database dominated world, built from parts scrounged from the remnants of enterprises and datacenters past. Twitter and Digg started in this era, but are evolving into something different, as scaling pressures increase and new purpose built technologies pop into being. With a little perspective, it's clear the MySQL+memcached era is passing. It will stick around for a while. Old technologies seldom fade away completely. Some still
4 0.94240451 702 high scalability-2009-09-11-The interactive cloud
Introduction: How many times have you been called in the middle of the night by your operation guys telling you that your application throws some odd red alerts? How many times did you found out that when those issues happens you don't have enough information to analyze this incident? have you tried to increase the log level just to find out that your problem became even worse - now your application throws tons of information in a continues basis most of which is complete garbage... The current separation between the way we implement our application and the way we manage it leads to many of this ridicules situations. Cloud makes those things even worse. In this post i suggest an alternative approach. Why don't we run our application the way we run our business? I refer to this approach as the "interactive cloud" where our application behaves just like our project team and the operations just like our managers. As with our business our application would need to take more responsibility to the wa
5 0.93895882 1255 high scalability-2012-06-01-Stuff The Internet Says On Scalability For June 1, 2012
Introduction: It's HighScalability Time: Yottabytes : What NSA knows about US; 214ms : ping between San Jose and Fez ; $42M : MongoDB is funding scale!; 20K : lines of THX sound code @adrianco : My takeaway from the MongoDB talk at #gluecon is that Mongo is implementing eventual scalability in the next version The death of the general purpose computer is causing strange events like Facebook making their own smart phone . Adam Smith said we all benefit when our neighbors get richer, it creates a bigger pie. We are heading back to the mercantalist notion of a zero sum game. Google is also racing to the bottom Google Product Search To Become Google Shopping, Use Pay-To-Play Model . Zero sum thinking always leads to war. Just sayin. Stuxnet , sometimes you just can't keep it in your pants and Pandora always complained that lid was never on very tight. Bad Prometheus. The Design of LLVM . Chris Lattner with a fascinating, detailed, and surpisingly clear look at LLVM, a univer
6 0.93635666 368 high scalability-2008-08-17-Wuala - P2P Online Storage Cloud
7 0.92323297 1020 high scalability-2011-04-12-Caching and Processing 2TB Mozilla Crash Reports in memory with Hazelcast
8 0.91214842 589 high scalability-2009-05-05-Drop ACID and Think About Data
9 0.91143626 1221 high scalability-2012-04-03-Hazelcast 2.0: Big Data In-Memory
10 0.9113133 1535 high scalability-2013-10-21-Google's Sanjay Ghemawat on What Made Google Google and Great Big Data Career Advice
11 0.91112316 671 high scalability-2009-08-05-Stack Overflow Architecture
12 0.91100395 1000 high scalability-2011-03-08-Medialets Architecture - Defeating the Daunting Mobile Device Data Deluge
13 0.91077065 1327 high scalability-2012-09-21-Stuff The Internet Says On Scalability For September 21, 2012
14 0.91041601 554 high scalability-2009-04-04-Digg Architecture
15 0.90971017 1460 high scalability-2013-05-17-Stuff The Internet Says On Scalability For May 17, 2013
16 0.90952158 1600 high scalability-2014-02-21-Stuff The Internet Says On Scalability For February 21st, 2014
17 0.90915537 849 high scalability-2010-06-28-VoltDB Decapitates Six SQL Urban Myths and Delivers Internet Scale OLTP in the Process
18 0.90891093 519 high scalability-2009-02-23-Database Sharding at Netlog, with MySQL and PHP
19 0.90767825 1131 high scalability-2011-10-24-StackExchange Architecture Updates - Running Smoothly, Amazon 4x More Expensive
20 0.90752906 687 high scalability-2009-08-24-How Google Serves Data from Multiple Datacenters