high_scalability high_scalability-2013 high_scalability-2013-1519 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Power on a cell phone is like water in a desert. It’s the very stuff of life. If you take the same naive programming techniques you learned when programming on a server in a datacenter your cell phone will die of thirst. This is dramatically shown by Reto Meier , Tech Lead for the Android Developer Relations Team, in a remarkable series of instructional videos: DevBytes: Efficient Data Transfers - Understanding the Cell Radio DevBytes: Efficient Data Transfers - Analyzing Your Transfer Profile DevBytes: Efficient Data Transfers - Effective Prefetching DevBytes: Efficient Data Transfers - Batching, Bundling, and SyncAdapters Remarkable because these videos are short, to the point, and chocked full of useful ideas and techniques. Though Android is targeted specifically, most of content should be generally useful. As an example of how server programming differs from mobile programming, on a server copying a file from one server
sentIndex sentText sentNum sentScore
1 You’ll learn that a bit later, but what you need to do is minimize radio usage by batching and properly scheduling transfers. [sent-15, score-0.676]
2 Key idea : The cell radio is one of the biggest battery drains on a phone. [sent-20, score-1.165]
3 To minimize radio usage and thus minimize power usage you must first understand how a cell radio works. [sent-27, score-1.718]
4 To send data the radio transitions to Full Power mode, a process that takes 2 seconds before performing the transfer. [sent-32, score-0.741]
5 For every data transfer the radio stays on for 5 seconds at full power followed by 10-60 seconds at a lower power state before returning to Standby. [sent-45, score-1.414]
6 Every time you transfer data you are powering the radio for at least 20 seconds. [sent-46, score-0.799]
7 The videos go into a number of techniques to bring about balance: Minimize the number of radio state transitions. [sent-53, score-0.726]
8 Determine your applications battery usage profile Generate graphs using: Logcat logging / Application Resource Optimizer / Network Statistics in DDMS Analyze graph for battery inefficiencies Look for regular pattern of transfers. [sent-54, score-0.803]
9 This will cause regular radio usage and power drain. [sent-55, score-0.753]
10 By downloading all the data a user is likely to need in a single burst over a single connection at full capacity, the number of radio activations is significantly reduced. [sent-60, score-0.774]
11 It’s more efficient to have transfers occur when a radio is in its active state, so if a time sensitive transfer is initiated, look to preempt, by transferring the data now, any transfers that will need to occur in a few minutes. [sent-68, score-1.283]
12 For a news reader, the naive way is to download top level and thumbnails, this keeps the radio constantly busy. [sent-72, score-0.677]
13 It transfers data in bursts rather than a continuous stream which would keep the radio constantly active. [sent-86, score-0.842]
14 These videos make clear what may have not been clear before: programming cell radio based mobile devices is a specialized domain that takes some specialized knowledge and techniques to do well. [sent-103, score-0.932]
15 dzamir : In fact iOS7 uses the same strategy described in this post: it wake ups all the applications that require a background download at the same time to minimize the time the radio is on. [sent-107, score-0.843]
16 If you know that using the mobile radio in a certain way is a source of excessive battery usage, it's silly to just disregard this information. [sent-122, score-0.991]
17 The mobile radio will eat your battery very quickly, and probably chews through as much power as your screen. [sent-124, score-1.141]
18 The keep-alive packets will keep your mobile radio in a higher power state, pretty much in the same way some disrespectful apps do and you'll (quite unsurprisingly) see your battery drain faster. [sent-127, score-1.316]
19 In the phone API's, if I need to do a background update, I should be able to register with the OS, and say, I have an update to do, within the next 15 minutes, let me know when you have an active radio connection. [sent-156, score-0.733]
20 You're phone goes into a high powered radio connection, and bam, all you're background updates now go out together. [sent-157, score-0.733]
wordName wordTfidf (topN-words)
[('radio', 0.532), ('battery', 0.366), ('cell', 0.216), ('transfers', 0.215), ('power', 0.15), ('prefetch', 0.138), ('app', 0.129), ('phone', 0.124), ('transfer', 0.118), ('state', 0.103), ('premature', 0.097), ('data', 0.095), ('downloading', 0.095), ('syncadapter', 0.093), ('mobile', 0.093), ('naive', 0.092), ('videos', 0.091), ('apps', 0.091), ('drain', 0.084), ('background', 0.077), ('seconds', 0.077), ('android', 0.073), ('minimize', 0.073), ('device', 0.071), ('usage', 0.071), ('minimizing', 0.067), ('carrier', 0.067), ('transitions', 0.066), ('network', 0.064), ('actually', 0.062), ('standby', 0.06), ('every', 0.06), ('life', 0.059), ('cookie', 0.058), ('invoice', 0.056), ('time', 0.054), ('efficient', 0.054), ('download', 0.053), ('full', 0.052), ('downloads', 0.051), ('prefetching', 0.051), ('drains', 0.051), ('radios', 0.051), ('optimization', 0.049), ('intermittent', 0.049), ('send', 0.048), ('balance', 0.048), ('office', 0.047), ('sent', 0.047), ('bundling', 0.046)]
simIndex simValue blogId blogTitle
same-blog 1 1.0000001 1519 high scalability-2013-09-18-If You're Programming a Cell Phone Like a Server You're Doing it Wrong
Introduction: Power on a cell phone is like water in a desert. It’s the very stuff of life. If you take the same naive programming techniques you learned when programming on a server in a datacenter your cell phone will die of thirst. This is dramatically shown by Reto Meier , Tech Lead for the Android Developer Relations Team, in a remarkable series of instructional videos: DevBytes: Efficient Data Transfers - Understanding the Cell Radio DevBytes: Efficient Data Transfers - Analyzing Your Transfer Profile DevBytes: Efficient Data Transfers - Effective Prefetching DevBytes: Efficient Data Transfers - Batching, Bundling, and SyncAdapters Remarkable because these videos are short, to the point, and chocked full of useful ideas and techniques. Though Android is targeted specifically, most of content should be generally useful. As an example of how server programming differs from mobile programming, on a server copying a file from one server
2 0.24562141 343 high scalability-2008-06-09-Apple's iPhone to Use a Centralized Push Based Notification Architecture
Introduction: Update 2: Hank Williams says iPhone Background Processing: Not Fixed But Halfway There . Excellent analysis of all the reasons you need real background processing. Hey, you can't even build an alarm clock! Hard to believe some commenters say it's not so. . Update: Josh Lowensohn of Webware tells us Why users should be scared of Apple's new notification system . A big item on the iPhone developer iWishlist has been background processing. If you can't write an app to poll for new data in the background how will you keep your even more important non-foreground app in sync? Live from the Apple developer conference we learn the solution is a centralized push based architecture. Here's the relevant MacRumorsLive transcript: Thanking the developers for their hard work. Now talking about how the #1 request has been background support. Apple wants to solve this problem. The wrong solution would be to allow for background processes -- bad for battery life and performance. Po
3 0.15804768 1242 high scalability-2012-05-09-Cell Architectures
Introduction: A consequence of Service Oriented Architectures is the burning need to provide services at scale. The architecture that has evolved to satisfy these requirements is a little known technique called the Cell Architecture. A Cell Architecture is based on the idea that massive scale requires parallelization and parallelization requires components be isolated from each other. These islands of isolation are called cells. A cell is a self-contained installation that can satisfy all the operations for a shard . A shard is a subset of a much larger dataset, typically a range of users, for example. Cell Architectures have several advantages: Cells provide a unit of parallelization that can be adjusted to any size as the user base grows. Cell are added in an incremental fashion as more capacity is required. Cells isolate failures. One cell failure does not impact other cells. Cells provide isolation as the storage and application horsepower to process requests is independent of othe
4 0.15390559 1191 high scalability-2012-02-13-Tumblr Architecture - 15 Billion Page Views a Month and Harder to Scale than Twitter
Introduction: With over 15 billion page views a month Tumblr has become an insanely popular blogging platform. Users may like Tumblr for its simplicity, its beauty, its strong focus on user experience, or its friendly and engaged community, but like it they do.Growing at over 30% a month has not been without challenges. Some reliability problems among them. It helps to realize that Tumblr operates at surprisingly huge scales: 500 million page views a day, a peak rate of ~40k requests per second, ~3TB of new data to store a day, all running on 1000+ servers.One of the common patterns across successful startups is the perilous chasm crossing from startup to wildly successful startup. Finding people, evolving infrastructures, servicing old infrastructures, while handling huge month over month increases in traffic, all with only four engineers, means you have to make difficult choices about what to work on. This was Tumblr's situation. Now with twenty engineers there's enough energy to work on issues an
Introduction: With over 15 billion page views a month Tumblr has become an insanely popular blogging platform. Users may like Tumblr for its simplicity, its beauty, its strong focus on user experience, or its friendly and engaged community, but like it they do.Growing at over 30% a month has not been without challenges. Some reliability problems among them. It helps to realize that Tumblr operates at surprisingly huge scales: 500 million page views a day, a peak rate of ~40k requests per second, ~3TB of new data to store a day, all running on 1000+ servers.One of the common patterns across successful startups is the perilous chasm crossing from startup to wildly successful startup. Finding people, evolving infrastructures, servicing old infrastructures, while handling huge month over month increases in traffic, all with only four engineers, means you have to make difficult choices about what to work on. This was Tumblr's situation. Now with twenty engineers there's enough energy to work on issues an
6 0.15277876 1461 high scalability-2013-05-20-The Tumblr Architecture Yahoo Bought for a Cool Billion Dollars
7 0.14788456 778 high scalability-2010-02-15-The Amazing Collective Compute Power of the Ambient Cloud
8 0.14290106 768 high scalability-2010-02-01-What Will Kill the Cloud?
11 0.13154849 1637 high scalability-2014-04-25-Stuff The Internet Says On Scalability For April 25th, 2014
12 0.12491932 1042 high scalability-2011-05-17-Facebook: An Example Canonical Architecture for Scaling Billions of Messages
13 0.12453487 1494 high scalability-2013-07-19-Stuff The Internet Says On Scalability For July 19, 2013
14 0.12334347 1000 high scalability-2011-03-08-Medialets Architecture - Defeating the Daunting Mobile Device Data Deluge
15 0.11858811 1240 high scalability-2012-05-07-Startups are Creating a New System of the World for IT
16 0.11399943 661 high scalability-2009-07-25-Latency is Everywhere and it Costs You Sales - How to Crush it
17 0.11261873 882 high scalability-2010-08-18-Misco: A MapReduce Framework for Mobile Systems - Start of the Ambient Cloud?
18 0.11200376 821 high scalability-2010-05-03-MocoSpace Architecture - 3 Billion Mobile Page Views a Month
topicId topicWeight
[(0, 0.214), (1, 0.091), (2, -0.009), (3, 0.012), (4, -0.022), (5, -0.032), (6, 0.034), (7, 0.095), (8, -0.003), (9, -0.026), (10, 0.006), (11, 0.052), (12, 0.008), (13, 0.01), (14, 0.037), (15, 0.058), (16, -0.052), (17, -0.021), (18, 0.001), (19, -0.055), (20, -0.067), (21, -0.016), (22, -0.022), (23, -0.017), (24, -0.013), (25, 0.015), (26, -0.029), (27, -0.014), (28, -0.015), (29, -0.03), (30, -0.016), (31, -0.005), (32, 0.079), (33, 0.015), (34, 0.007), (35, 0.022), (36, 0.049), (37, 0.037), (38, 0.034), (39, -0.056), (40, 0.013), (41, -0.007), (42, -0.028), (43, -0.031), (44, 0.051), (45, -0.008), (46, -0.029), (47, 0.07), (48, -0.055), (49, -0.027)]
simIndex simValue blogId blogTitle
same-blog 1 0.92157471 1519 high scalability-2013-09-18-If You're Programming a Cell Phone Like a Server You're Doing it Wrong
Introduction: Power on a cell phone is like water in a desert. It’s the very stuff of life. If you take the same naive programming techniques you learned when programming on a server in a datacenter your cell phone will die of thirst. This is dramatically shown by Reto Meier , Tech Lead for the Android Developer Relations Team, in a remarkable series of instructional videos: DevBytes: Efficient Data Transfers - Understanding the Cell Radio DevBytes: Efficient Data Transfers - Analyzing Your Transfer Profile DevBytes: Efficient Data Transfers - Effective Prefetching DevBytes: Efficient Data Transfers - Batching, Bundling, and SyncAdapters Remarkable because these videos are short, to the point, and chocked full of useful ideas and techniques. Though Android is targeted specifically, most of content should be generally useful. As an example of how server programming differs from mobile programming, on a server copying a file from one server
2 0.82020986 343 high scalability-2008-06-09-Apple's iPhone to Use a Centralized Push Based Notification Architecture
Introduction: Update 2: Hank Williams says iPhone Background Processing: Not Fixed But Halfway There . Excellent analysis of all the reasons you need real background processing. Hey, you can't even build an alarm clock! Hard to believe some commenters say it's not so. . Update: Josh Lowensohn of Webware tells us Why users should be scared of Apple's new notification system . A big item on the iPhone developer iWishlist has been background processing. If you can't write an app to poll for new data in the background how will you keep your even more important non-foreground app in sync? Live from the Apple developer conference we learn the solution is a centralized push based architecture. Here's the relevant MacRumorsLive transcript: Thanking the developers for their hard work. Now talking about how the #1 request has been background support. Apple wants to solve this problem. The wrong solution would be to allow for background processes -- bad for battery life and performance. Po
3 0.73030585 960 high scalability-2010-12-20-Netflix: Use Less Chatty Protocols in the Cloud - Plus 26 Fixes
Introduction: In 5 Lessons We’ve Learned Using AWS , Netflix's John Ciancutti says one of the big lessons they've learned is to create less chatty protocols : In the Netflix data centers, we have a high capacity, super fast, highly reliable network. This has afforded us the luxury of designing around chatty APIs to remote systems. AWS networking has more variable latency. We’ve had to be much more structured about “over the wire” interactions, even as we’ve transitioned to a more highly distributed architecture. There's not a lot of advice out there on how to create protocols. Combine that with a rush to the cloud and you have a perfect storm for chatty applications crushing application performance. Netflix is far from the first to be surprised by the less than stellar networks inside AWS. A chatty protocol is one where a client makes a series of requests to a server and the client must wait on each reply before sending the next request. On a LAN this can work great. LAN's are typically
Introduction: If you were going to design a next generation Internet at the physical layer that routes around the current Internet, what would it look like? What should it do? How should it work? Who should own it? How should it be paid for? How would you access it? It has long been said the Internet routes around obstacles. Snowden has revealed some major obstacles. The beauty of the current current app and web system is the physical network doesn't matter. We can just replace it with something else. Something that doesn't flow through choke points like backhaul networks , under sea cables , and cell towers . What might that something else look like? Google's Loon Project Project Loon was so named because the idea was thought to be loony. Maybe not. The idea is to float high-altitude balloons 20 miles in the air to create an aerial wireless network with up to 3G-like speeds. Signals travel through the balloon network from balloon to balloon, then to a ground-based station conne
5 0.70951068 1612 high scalability-2014-03-14-Stuff The Internet Says On Scalability For March 14th, 2014
Introduction: Hey, it's HighScalability time: LifeExplorer Cells in 3D Quotable Quotes: The Master Switch : History shows a typical progression of information technologies: from somebody’s hobby to somebody’s industry; from jury-rigged contraption to slick production marvel; from a freely accessible channel to one strictly controlled by a single corporation or cartel—from open to closed system. @adrianco : #qconlondon @russmiles on PaaS "As old as I am, a leaky abstraction would be awful..." @Obdurodon : "Scaling is hard. Let's make excuses." @TomRoyce : @jeffjarvis the rot is deep... The New Jersey pols just used Tesla to shake down the car dealers. @CompSciFact : "The cheapest, fastest and most reliable components of a computer system are those that aren't there." -- Gordon Bell @glyph : “Eventually consistent” is just another way to say “not consistent right now”. @nutshell : LinkedIn is shutting down access to their APIs
6 0.70631272 1225 high scalability-2012-04-09-Why My Slime Mold is Better than Your Hadoop Cluster
7 0.70169181 1091 high scalability-2011-08-02-How Will DIDO Wireless Networking Change Everything?
8 0.69889927 1472 high scalability-2013-06-07-Stuff The Internet Says On Scalability For June 7, 2013
9 0.69717324 1600 high scalability-2014-02-21-Stuff The Internet Says On Scalability For February 21st, 2014
10 0.69451505 1630 high scalability-2014-04-11-Stuff The Internet Says On Scalability For April 11th, 2014
11 0.69392943 1259 high scalability-2012-06-07-3 Secrets to Lightning Fast Mobile Design at Instagram
12 0.69260514 1572 high scalability-2014-01-03-Stuff The Internet Says On Scalability For January 3rd, 2014
13 0.68849218 1634 high scalability-2014-04-18-Stuff The Internet Says On Scalability For April 18th, 2014
15 0.68483907 1592 high scalability-2014-02-07-Stuff The Internet Says On Scalability For February 7th, 2014
16 0.68226451 1494 high scalability-2013-07-19-Stuff The Internet Says On Scalability For July 19, 2013
17 0.68175411 976 high scalability-2011-01-20-75% Chance of Scale - Leveraging the New Scaleogenic Environment for Growth
18 0.68000531 1190 high scalability-2012-02-10-Stuff The Internet Says On Scalability For February 10, 2012
19 0.67849571 1637 high scalability-2014-04-25-Stuff The Internet Says On Scalability For April 25th, 2014
20 0.67771471 1001 high scalability-2011-03-09-Google and Netflix Strategy: Use Partial Responses to Reduce Request Sizes
topicId topicWeight
[(1, 0.146), (2, 0.205), (10, 0.04), (22, 0.011), (30, 0.031), (40, 0.024), (49, 0.025), (61, 0.09), (79, 0.117), (85, 0.053), (88, 0.106), (94, 0.037), (96, 0.011)]
simIndex simValue blogId blogTitle
1 0.95109993 1524 high scalability-2013-09-30-Expandability: Steve Wozniak's Biggest Success and Nolan Bushnell's Biggest Regret
Introduction: When two titans of the early computing industry, Apple co-founder Steve Wozniak and Atari founder Nolan Bushnell , had a chance to sit down for a chat at the C2SV conference last Friday, we learned how a little foresight and luck can decide futures. Steve Wozniak's Success - Expandability Steve talked about how the Apple II was the primary source of revenue for Apple's first 10 years of life. The reason the Apple II could last so long was that it was expandable. It had eight expansion slots and the RAM was also expandable (starting at 4K). No computer at the time had this kind of flexibility or expansion capabilities . Steve Wozniak and Steve Jobs argued over keeping the Apple II open. Jobs wanted closed, Wozniak wanted open and fortunately Wozniak won. In fact, the Apple II was completely open (those were the days). They published the complete specs before there was anything like a developer community. So even though the Apple III, Lisa, and the Macintosh al
same-blog 2 0.94822824 1519 high scalability-2013-09-18-If You're Programming a Cell Phone Like a Server You're Doing it Wrong
Introduction: Power on a cell phone is like water in a desert. It’s the very stuff of life. If you take the same naive programming techniques you learned when programming on a server in a datacenter your cell phone will die of thirst. This is dramatically shown by Reto Meier , Tech Lead for the Android Developer Relations Team, in a remarkable series of instructional videos: DevBytes: Efficient Data Transfers - Understanding the Cell Radio DevBytes: Efficient Data Transfers - Analyzing Your Transfer Profile DevBytes: Efficient Data Transfers - Effective Prefetching DevBytes: Efficient Data Transfers - Batching, Bundling, and SyncAdapters Remarkable because these videos are short, to the point, and chocked full of useful ideas and techniques. Though Android is targeted specifically, most of content should be generally useful. As an example of how server programming differs from mobile programming, on a server copying a file from one server
3 0.94044238 1348 high scalability-2012-10-26-Stuff The Internet Says On Scalability For October 26, 2012
Introduction: It's HighScalability Time: 1.5 Billion Pageviews : Etsy in September; 200 dedicated database servers : Tumblr Quotable Quotes: @rbranson : Datadog stays available where it counts (metrics injest) by using Cassandra, combined with an RDBMS for queries. Nice. @jmhodges : Few engineers know what modern hw is capable of, in part, because the only people that see the numbers are in orgs that had to care or die. @tinagroves : Storing the brain in the cloud might cost $38/month asserts Jim Adelius at #strataconf in talk on #bigdata and thought crimes. Why is it hard to scale a database, in layman’s terms? on Quora. Some really good answers. My answer would involve a cookie jar filled with all different kinds of cookies and a motley crew of kindergartners all trying to get cookies at the same time while keebler elves are trying to fill up the jar, all at the same time. Rackspace now has their own cloud block storage prod
4 0.93818814 1339 high scalability-2012-10-12-Stuff The Internet Says On Scalability For October 12, 2012
Introduction: It's HighScalability Time: Quotable Quotes: @rodos : With so much manure in the room of #BigData the must be a pony in here somewhere! @adron : #ricon2012 OH "it isn't cloud, nosql or whatever, it's distributed systems... That is the change!" Smartest thing stated at a conf in ages. @justinesherry : First lesson watching @eric_brewer's keynote at #ricon2012 -- distributed systems make you go bald. (Both audience and speaker!) @adron : OH "What's the hello world of distributed systems?" "Twitter." Mikael Ronstrom : A long career on distributed systems has learnt me that it is extremely important to do proper partitioning of data sets to achieve scalability on network level. But it is still extremely important to make each node in the distributed system as large as possible. Greg Linden : Surprising: "It can be three to four times the cost for the retailer to pick items and pack them from a store versus having a really e
5 0.93351221 1269 high scalability-2012-06-20-iDoneThis - Scaling an Email-based App from Scratch
Introduction: This is a guest post by Rodrigo Guzman, CTO of iDoneThis , which makes status reporting happen at your company with the lightest possible touch. iDoneThis is a simple management application that emails your team at the end of every day to ask, "What'd you get done today?" Just reply with a few lines of what you got done. The following morning everyone on your team gets a digest with what the team accomplished the previous day to keep everyone in the loop and kickstart another awesome day. Before we launched, we built iDoneThis over a weekend in the most rudimentary way possible. I kid you not, we sent the first few batches of daily emails using the BCC field of a Gmail inbox. The upshot is that we’ve had users on the site from Day 3 of its existence on. We’ve gone from launch in January 2011 when we sent hundreds of emails out per day by hand to sending out over 1 million emails and handling over 200,000 incoming emails per month. In total, customers have recorded over 1.
6 0.93323201 1334 high scalability-2012-10-04-Stuff The Internet Says On Scalability For October 5, 2012
7 0.93184751 1600 high scalability-2014-02-21-Stuff The Internet Says On Scalability For February 21st, 2014
8 0.93163365 1431 high scalability-2013-03-29-Stuff The Internet Says On Scalability For March 29, 2013
10 0.92982233 1502 high scalability-2013-08-16-Stuff The Internet Says On Scalability For August 16, 2013
11 0.92926753 1407 high scalability-2013-02-15-Stuff The Internet Says On Scalability For February 15, 2013
12 0.92914379 857 high scalability-2010-07-13-DbShards Part Deux - The Internals
13 0.9291417 776 high scalability-2010-02-12-Hot Scalability Links for February 12, 2010
14 0.92901891 1052 high scalability-2011-06-03-Stuff The Internet Says On Scalability For June 3, 2011
15 0.92892456 1559 high scalability-2013-12-06-Stuff The Internet Says On Scalability For December 6th, 2013
16 0.92884886 1389 high scalability-2013-01-18-Stuff The Internet Says On Scalability For January 18, 2013
17 0.9286992 195 high scalability-2007-12-28-Amazon's EC2: Pay as You Grow Could Cut Your Costs in Half
19 0.92774177 1008 high scalability-2011-03-22-Facebook's New Realtime Analytics System: HBase to Process 20 Billion Events Per Day
20 0.92758381 1302 high scalability-2012-08-10-Stuff The Internet Says On Scalability For August 10, 2012