high_scalability high_scalability-2011 high_scalability-2011-1090 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: This is a guest post by Marcel Panse and Sander Nagtegaal from Peecho . Although architecture descriptions are an interesting read, the problems that start-ups face are hardly ever addressed. We would like to change that, so here is our architecture story. Introducing a start-up The Amsterdam-based company Peecho offers print-as-a-service. Our embeddable print button allows you to sell your digital content as professionally printed products, like photo books, magazines or canvases - straight from your own website. There is an API, too. Printcloud is the system that powers the print button. It exists in the cloud only, growing when needed and becoming smaller if it can. The system takes in print orders, magically transforms tough data into print-ready files and routes the orders to the production facility that is closest to the intended recipient. To preserve the environment, Peecho's philosophy is to facilitate global ordering, but to aim for local production on
sentIndex sentText sentNum sentScore
1 Our embeddable print button allows you to sell your digital content as professionally printed products, like photo books, magazines or canvases - straight from your own website. [sent-5, score-0.657]
2 Printcloud is the system that powers the print button. [sent-7, score-0.365]
3 The system takes in print orders, magically transforms tough data into print-ready files and routes the orders to the production facility that is closest to the intended recipient. [sent-9, score-0.935]
4 However, we needed queueing and relational data storage - exit Google. [sent-20, score-0.15]
5 Scalability for small companies is largely obtained by flexibility. [sent-26, score-0.152]
6 As a result, we can afford to only build the bare minimum that is needed without thinking much about the future needs. [sent-29, score-0.175]
7 That's when demos are shown and everybody cheers and applauds for the new stuff. [sent-34, score-0.173]
8 The print button and its checkout exist as a separate application on top of our platform. [sent-40, score-0.632]
9 It stores order data in SimpleDB, an AWS No-SQL data store that can take a beating. [sent-43, score-0.132]
10 After payment has been confirmed, the print button checkout submits every order to a REST API. [sent-45, score-0.838]
11 The order intake machine writes a ticket to the processing queue. [sent-50, score-0.375]
12 Whenever there are enough tickets available, a new processing machine wakes up, gets a ticket and starts crunching that awful data. [sent-51, score-0.375]
13 So, using queue metrics, we scale the amount of processing power up and down as the number of items in the queue varies. [sent-54, score-0.226]
14 Subsequently, the order must be routed to the right production facility by adding a production ticket to the right print facility queue. [sent-58, score-1.424]
15 Using either our Adobe AIR Printclient desktop application or a direct integration, the facility retrieves its jobs from the queue and the product files from S3 file storage. [sent-59, score-0.533]
16 In return, the facility can update job statuses in Printcloud by posting messages to the central order status queue. [sent-61, score-0.488]
17 After a certain period of time, the product files will be removed from S3 to save storage costs - and everybody lives happily ever after. [sent-64, score-0.237]
18 Spot instances are generally a bargain and we assume a continuous 50% workload. [sent-70, score-0.326]
19 Just like the number of concurrent EC2 nodes, required bandwidth is largely determined by the amount of orders coming in. [sent-71, score-0.197]
20 $ 50 That makes a total of 486 dollars a month for a fully elastic, heavy duty system that has the power of a minimum of 5 servers, 2 database servers, a No-SQL data store and external flat file storage. [sent-73, score-0.155]
wordName wordTfidf (topN-words)
[('print', 0.365), ('facility', 0.278), ('simpledb', 0.186), ('ticket', 0.165), ('button', 0.148), ('order', 0.132), ('rds', 0.127), ('instances', 0.124), ('checkout', 0.119), ('orders', 0.113), ('queue', 0.113), ('production', 0.103), ('sqs', 0.1), ('everybody', 0.099), ('minimum', 0.089), ('needed', 0.086), ('largely', 0.084), ('professionally', 0.078), ('prepares', 0.078), ('intake', 0.078), ('statuses', 0.078), ('peecho', 0.078), ('shortlist', 0.078), ('wakes', 0.078), ('files', 0.076), ('stuff', 0.075), ('submits', 0.074), ('cheers', 0.074), ('iam', 0.074), ('aws', 0.072), ('bucks', 0.07), ('marcel', 0.07), ('bargain', 0.07), ('guns', 0.07), ('cohesion', 0.07), ('continuous', 0.068), ('obtained', 0.068), ('awful', 0.068), ('atlassian', 0.068), ('duty', 0.066), ('retrieves', 0.066), ('printed', 0.066), ('confirmed', 0.066), ('generally', 0.064), ('exit', 0.064), ('adobe', 0.064), ('crunching', 0.064), ('component', 0.063), ('happily', 0.062), ('shipped', 0.062)]
simIndex simValue blogId blogTitle
same-blog 1 1.0 1090 high scalability-2011-08-01-Peecho Architecture - scalability on a shoestring
Introduction: This is a guest post by Marcel Panse and Sander Nagtegaal from Peecho . Although architecture descriptions are an interesting read, the problems that start-ups face are hardly ever addressed. We would like to change that, so here is our architecture story. Introducing a start-up The Amsterdam-based company Peecho offers print-as-a-service. Our embeddable print button allows you to sell your digital content as professionally printed products, like photo books, magazines or canvases - straight from your own website. There is an API, too. Printcloud is the system that powers the print button. It exists in the cloud only, growing when needed and becoming smaller if it can. The system takes in print orders, magically transforms tough data into print-ready files and routes the orders to the production facility that is closest to the intended recipient. To preserve the environment, Peecho's philosophy is to facilitate global ordering, but to aim for local production on
2 0.16961944 498 high scalability-2009-01-20-Product: Amazon's SimpleDB
Introduction: Update 35 : How and Why Glue is Using Amazon SimpleDB instead of a Relational Database . Discusses a key design decision that required duplicating data in order to mimic RDBMS joins: Given the trade off between potential inconsistencies and scalability, social services have to choose the latter. Update 34 : Apparently Amazon pulled this article. I'm not sure what that means. Maybe time went backwards or something? Amazon dramatically drops SimpleDB pricing to $0.25 per GB per month from $1.50 per GB . This puts SimpleDB on par with Google App Engine . They also announced a few new features: a SQL-like SELECT API as well as a Batch Put operation to streamline uploading of multiple items or attributes . One of the complaints against SimpleDB is that programmers end up writing too much code to do simple things. These features and a much cheaper price should help considerably. And you can store lots of data now. GAE is still capped. Update 33 : Amazon announces
3 0.15813409 674 high scalability-2009-08-07-The Canonical Cloud Architecture
Introduction: Update 2: Elastic Load Balancer and EC2 instance bandwidth . It turns out we are limited by bandwidth and not by CPU . Solution: use DNS Round Robin for two to three HighCPU medium instances . Update: The Skinny Straw: Cloud Computing's Bottleneck and How to Address It . For cloud computing, bandwidth to and from the cloud provider is a bottleneck . Solution: Evaluate application architecture and consider application partitioning . I'm writing this post as a sort of penance. My sin was getting involved in another mutli-threaded mess of a program that was rife with strange pauses and unexpected errors. I really should have known better. But when APIs choose to make callbacks from some mystery thread pool it's hard to keep things straight. I eventually sobered up and posted all events to a queue so I could make sure the program would work correctly. Doh. I may never know why the .Net console output stopped working, but I'll live with it. And that reminded me that I've been m
4 0.14849229 306 high scalability-2008-04-21-The Search for the Source of Data - How SimpleDB Differs from a RDBMS
Introduction: Update 2: Yurii responds with the Top 10 Reasons to Avoid Document Databases FUD . Update: Top 10 Reasons to Avoid the SimpleDB Hype by Ryan Park provides a well written counter take. Am I really that fawning? If so, doesn't that make me a dear? All your life you've used a relational database. At the tender age of five you banged out your first SQL query to track your allowance. Your RDBMS allegiance was just assumed, like your politics or religion would have been assumed 100 years ago. They now say--you know them--that relations won't scale and we have to do things differently. New databases like SimpleDB and BigTable are what's different. As a long time RDBMS user what can you expect of SimpleDB? That's what Alex Tolley of MyMeemz.com set out to discover. Like many brave explorers before him, Alex gave a report of his adventures to the Royal Society of the AWS Meetup . Alex told a wild almost unbelievable tale of cultures and practices so different from our own you alm
5 0.14781462 538 high scalability-2009-03-16-Are Cloud Based Memory Architectures the Next Big Thing?
Introduction: We are on the edge of two potent technological changes: Clouds and Memory Based Architectures. This evolution will rip open a chasm where new players can enter and prosper. Google is the master of disk. You can't beat them at a game they perfected. Disk based databases like SimpleDB and BigTable are complicated beasts, typical last gasp products of any aging technology before a change. The next era is the age of Memory and Cloud which will allow for new players to succeed. The tipping point will be soon. Let's take a short trip down web architecture lane: It's 1993: Yahoo runs on FreeBSD, Apache, Perl scripts and a SQL database It's 1995: Scale-up the database. It's 1998: LAMP It's 1999: Stateless + Load Balanced + Database + SAN It's 2001: In-memory data-grid. It's 2003: Add a caching layer. It's 2004: Add scale-out and partitioning. It's 2005: Add asynchronous job scheduling and maybe a distributed file system. It's 2007: Move it all into the cloud. It's 2008: C
6 0.14497042 184 high scalability-2007-12-13-Amazon SimpleDB - Scalable Cloud Database
7 0.14414914 1240 high scalability-2012-05-07-Startups are Creating a New System of the World for IT
8 0.14316514 925 high scalability-2010-10-22-Paper: Netflix’s Transition to High-Availability Storage Systems
9 0.14164571 38 high scalability-2007-07-30-Build an Infinitely Scalable Infrastructure for $100 Using Amazon Services
11 0.13881126 750 high scalability-2009-12-16-Building Super Scalable Systems: Blade Runner Meets Autonomic Computing in the Ambient Cloud
12 0.13728699 1331 high scalability-2012-10-02-An Epic TripAdvisor Update: Why Not Run on the Cloud? The Grand Experiment.
13 0.13438767 96 high scalability-2007-09-18-Amazon Architecture
14 0.13217744 1271 high scalability-2012-06-25-StubHub Architecture: The Surprising Complexity Behind the World’s Largest Ticket Marketplace
15 0.12857538 317 high scalability-2008-05-10-Hitting 300 SimbleDB Requests Per Second on a Small EC2 Instance
16 0.12445396 365 high scalability-2008-08-16-Strategy: Serve Pre-generated Static Files Instead Of Dynamic Pages
20 0.11916079 1112 high scalability-2011-09-07-What Google App Engine Price Changes Say About the Future of Web Architecture
topicId topicWeight
[(0, 0.249), (1, 0.063), (2, -0.002), (3, 0.012), (4, -0.047), (5, -0.052), (6, 0.039), (7, -0.079), (8, 0.021), (9, -0.091), (10, 0.003), (11, -0.018), (12, 0.021), (13, -0.075), (14, 0.015), (15, -0.017), (16, -0.043), (17, 0.014), (18, 0.02), (19, -0.009), (20, -0.015), (21, -0.025), (22, 0.047), (23, 0.003), (24, 0.01), (25, -0.059), (26, 0.039), (27, 0.022), (28, 0.051), (29, -0.002), (30, -0.031), (31, 0.016), (32, -0.003), (33, -0.012), (34, -0.027), (35, -0.022), (36, -0.024), (37, 0.003), (38, -0.003), (39, -0.035), (40, 0.002), (41, -0.053), (42, -0.016), (43, -0.008), (44, -0.007), (45, 0.014), (46, -0.009), (47, -0.039), (48, -0.045), (49, -0.03)]
simIndex simValue blogId blogTitle
same-blog 1 0.96206892 1090 high scalability-2011-08-01-Peecho Architecture - scalability on a shoestring
Introduction: This is a guest post by Marcel Panse and Sander Nagtegaal from Peecho . Although architecture descriptions are an interesting read, the problems that start-ups face are hardly ever addressed. We would like to change that, so here is our architecture story. Introducing a start-up The Amsterdam-based company Peecho offers print-as-a-service. Our embeddable print button allows you to sell your digital content as professionally printed products, like photo books, magazines or canvases - straight from your own website. There is an API, too. Printcloud is the system that powers the print button. It exists in the cloud only, growing when needed and becoming smaller if it can. The system takes in print orders, magically transforms tough data into print-ready files and routes the orders to the production facility that is closest to the intended recipient. To preserve the environment, Peecho's philosophy is to facilitate global ordering, but to aim for local production on
2 0.87485254 38 high scalability-2007-07-30-Build an Infinitely Scalable Infrastructure for $100 Using Amazon Services
Introduction: Can you really create an infinitely scalable infrastructure for less than $100 using Amazon's storage, grid, and queuing services platform? It appears so, at least for the right application. Amazon beams a spot light on the future battle of the roll-your-own versus the connect-the-dots approach to building next generation websites using core external services. Their argument is strong. Using Amazon's platform you can quickly build an infrastructure that would otherwise take an eternity to make, a pile of money to create, and an unbounded mass of people to implement and maintain. Yet Amazon doesn't provide SLAs, so you can you really trust them with your crown jewels? Facebook recently leap frogged Amazon's vision with an even more comprehensive set of services. The battle for the future is on. Site: http://aws.amazon.com/ Information Sources Slides: Building Highly Scalable Web Applications Podcast: Technometria: Amazon Web Services Amazon Services Home . Platform
3 0.85254371 674 high scalability-2009-08-07-The Canonical Cloud Architecture
Introduction: Update 2: Elastic Load Balancer and EC2 instance bandwidth . It turns out we are limited by bandwidth and not by CPU . Solution: use DNS Round Robin for two to three HighCPU medium instances . Update: The Skinny Straw: Cloud Computing's Bottleneck and How to Address It . For cloud computing, bandwidth to and from the cloud provider is a bottleneck . Solution: Evaluate application architecture and consider application partitioning . I'm writing this post as a sort of penance. My sin was getting involved in another mutli-threaded mess of a program that was rife with strange pauses and unexpected errors. I really should have known better. But when APIs choose to make callbacks from some mystery thread pool it's hard to keep things straight. I eventually sobered up and posted all events to a queue so I could make sure the program would work correctly. Doh. I may never know why the .Net console output stopped working, but I'll live with it. And that reminded me that I've been m
4 0.8437376 798 high scalability-2010-03-22-7 Secrets to Successfully Scaling with Scalr (on Amazon) by Sebastian Stadil
Introduction: This is a part interview part guest with Sebastian Stadil, founder of Scalr , a cheaper open-source version of RightScale. Scalr takes care of all the web site infrastructure bits to on Amazon (and other clouds) so you don’t have to. I first met Sebastian at one of the original Silicon Valley Cloud Computing Group meetups, a group which he founded. The meetings started in the tiny offices of Intalio where Sebastian was working with this new fangled Amazon thing to create an auto-scaling server farm on EC2. I remember distinctly how Sebastian met everyone at the door with a handshake and a smile, making us all feel welcome. Later I took one of the classes he created on how to use AWS. I guess he figured all this cloud stuff was going somewhere and decided to start Scalr. My only regret about this post is that the name Amazon does not begin with the letter ‘S’, that would have made for an epic title. Getting to Know You In this section are the responses Sebastian gave to a few
5 0.79847229 96 high scalability-2007-09-18-Amazon Architecture
Introduction: This is a wonderfully informative Amazon update based on Joachim Rohde's discovery of an interview with Amazon's CTO. You'll learn about how Amazon organizes their teams around services, the CAP theorem of building scalable systems, how they deploy software, and a lot more. Many new additions from the ACM Queue article have also been included. Amazon grew from a tiny online bookstore to one of the largest stores on earth. They did it while pioneering new and interesting ways to rate, review, and recommend products. Greg Linden shared is version of Amazon's birth pangs in a series of blog articles Site: http://amazon.com Information Sources Early Amazon by Greg Linden How Linux saved Amazon millions Interview Werner Vogels - Amazon's CTO Asynchronous Architectures - a nice summary of Werner Vogels' talk by Chris Loosley Learning from the Amazon technology platform - A Conversation with Werner Vogels Werner Vogels' Weblog - building scalable and robus
6 0.79566962 498 high scalability-2009-01-20-Product: Amazon's SimpleDB
7 0.79451299 1557 high scalability-2013-12-02-Evolution of Bazaarvoice’s Architecture to 500M Unique Users Per Month
9 0.78861552 1106 high scalability-2011-08-26-Stuff The Internet Says On Scalability For August 26, 2011
10 0.78594911 1482 high scalability-2013-06-26-Leveraging Cloud Computing at Yelp - 102 Million Monthly Vistors and 39 Million Reviews
11 0.78485811 1470 high scalability-2013-06-05-A Simple 6 Step Transition Guide for Moving Away from X to AWS
12 0.77665353 1452 high scalability-2013-05-06-7 Not So Sexy Tips for Saving Money On Amazon
13 0.77169251 1331 high scalability-2012-10-02-An Epic TripAdvisor Update: Why Not Run on the Cloud? The Grand Experiment.
14 0.75301921 1549 high scalability-2013-11-15-Stuff The Internet Says On Scalability For November 15th, 2013
15 0.74695623 1197 high scalability-2012-02-21-Pixable Architecture - Crawling, Analyzing, and Ranking 20 Million Photos a Day
16 0.74487156 881 high scalability-2010-08-16-Scaling an AWS infrastructure - Tools and Patterns
17 0.74471146 1028 high scalability-2011-04-22-Stuff The Internet Says On Scalability For April 22, 2011
18 0.74411047 853 high scalability-2010-07-08-Cloud AWS Infrastructure vs. Physical Infrastructure
19 0.7433297 1198 high scalability-2012-02-24-Stuff The Internet Says On Scalability For February 24, 2012
20 0.74273258 663 high scalability-2009-07-28-37signals Architecture
topicId topicWeight
[(1, 0.149), (2, 0.197), (10, 0.072), (22, 0.135), (26, 0.011), (30, 0.032), (47, 0.017), (61, 0.075), (73, 0.022), (77, 0.018), (79, 0.108), (85, 0.042), (94, 0.037)]
simIndex simValue blogId blogTitle
Introduction: The story of driving the golden spike to symbolize the completion of the transcontinental railroad is famous in the US. What is not so well known is the story of how it also foreshadowed changes to come as an early version of both the Internet and the Internet of Things. But that was 1869, how can that possibly be? Telegraph as Internet First, let's establish the telegraph and cable systems was an early version of an Internet. As railroad tracks were being laid a transcontental telegraph system was also being constructed. Telegraph lines were installed parallel to the tracks making instant communication available across the continent, faster than any horse could ride. With the transalantic cable system information could quickly span continents in minutes: The miles of American telegraph grew from 40 in 1846 to 12,000 in 1850 to 23,000 in 1852. In Europe it increased from 2,000 in 1849 to 110,000 in 1869. The cost of sending 10 words was $1.55 in 1850, $1 in 1870, 40
2 0.94406891 1645 high scalability-2014-05-09-Stuff The Internet Says On Scalability For May 9th, 2014
Introduction: Hey, it's HighScalability time: NASA captures Guatemala volcano erupting from space 40,000 exabytes : from now until 2020, the digital universe will about double every two years; $650,000 : amount raised by the MaydayPAC in one week. Quotable Quotes: @BenedictEvans : Masayoshi Son: $20m initial investment in Alibaba, current stake worth $58bn. @iamdevloper : I sneezed earlier and Siri compiled it to valid Perl. @cdixon : "There is not enough competition in the last mile market to allow a true market to function" @PatrickMcFadin : Get ready for some serious server density. AMD is working on K12, brand-new x86 and ARM cores. This plus 8T SSD? With age comes changing priorities. Facebook is now 10 and has grown up . They are no longer moving fast and breaking things. They are now into the stability thing. Letting developers know they are a stable platform. The play is to get all that beautiful data from developers by bei
3 0.94266403 1543 high scalability-2013-11-05-10 Things You Should Know About AWS
Introduction: Authored by Chris Fregly : Former Netflix Streaming Platform Engineer, AWS Certified Solution Architect and Purveyor of fluxcapacitor.com. Ahead of the upcoming 2nd annual re:Invent conference, inspired by Simone Brunozzi’s recent presentation at an AWS Meetup in San Francisco, and collected from a few of my recent Fluxcapacitor.com consulting engagements, I’ve compiled a list of 10 useful time and clock-tick saving tips about AWS. 1) Query AWS resource metadata Can’t remember the EBS-Optimized IO throughput of your c1.xlarge cluster? How about the size limit of an S3 object on a single PUT? awsnow.info is the answer to all of your AWS-resource metadata questions. Interested in integrating awsnow.info with your application? You’re in luck. There’s now a REST API , as well! Note: These are default soft limits and will vary by account. 2) Tame your S3 buckets Delete an entire S3 bucket with a single CLI command:
same-blog 4 0.94109201 1090 high scalability-2011-08-01-Peecho Architecture - scalability on a shoestring
Introduction: This is a guest post by Marcel Panse and Sander Nagtegaal from Peecho . Although architecture descriptions are an interesting read, the problems that start-ups face are hardly ever addressed. We would like to change that, so here is our architecture story. Introducing a start-up The Amsterdam-based company Peecho offers print-as-a-service. Our embeddable print button allows you to sell your digital content as professionally printed products, like photo books, magazines or canvases - straight from your own website. There is an API, too. Printcloud is the system that powers the print button. It exists in the cloud only, growing when needed and becoming smaller if it can. The system takes in print orders, magically transforms tough data into print-ready files and routes the orders to the production facility that is closest to the intended recipient. To preserve the environment, Peecho's philosophy is to facilitate global ordering, but to aim for local production on
5 0.93071032 1621 high scalability-2014-03-28-Stuff The Internet Says On Scalability For March 28th, 2014
Introduction: Hey, it's HighScalability time: Looks like a multiverse , if you can keep it. Quotable Quotes: @abt_programming : "I am a Unix Creationist. I believe the world was created on January 1, 1970 and as prophesized, will end on January 19, 2038" - @teropa @demisbellot : Cloud prices are hitting attractive price points, one more 40-50% drop and there'd be little reason to go it alone. @scott4arrows : Dentist "Do you floss regularly?" Me "Do you back up your computer data regularly?" @avestal : "I Kickstarted the Oculus Rift, what do I get?" You get a lesson in how capitalism works. @mtabini : “$20 charger that cost $2 to make.” Not pictured here: the $14 you pay for the 10,000 charger iterations that never made it to production. @strlen : "I built the original assembler in JS, because it's what I prefer to use when I need to get down to bare metal." - Adm. Grace Hopper tedchs : I'd like to propose a new rule for Hacker News: only if you have b
6 0.92624545 774 high scalability-2010-02-08-How FarmVille Scales to Harvest 75 Million Players a Month
7 0.91829628 491 high scalability-2009-01-13-Product: Gearman - Open Source Message Queuing System
8 0.91294658 301 high scalability-2008-04-08-Google AppEngine - A First Look
9 0.90830326 1508 high scalability-2013-08-28-Sean Hull's 20 Biggest Bottlenecks that Reduce and Slow Down Scalability
10 0.89698082 1109 high scalability-2011-09-02-Stuff The Internet Says On Scalability For September 2, 2011
11 0.89668763 857 high scalability-2010-07-13-DbShards Part Deux - The Internals
12 0.89555752 881 high scalability-2010-08-16-Scaling an AWS infrastructure - Tools and Patterns
13 0.89551377 1553 high scalability-2013-11-25-How To Make an Infinitely Scalable Relational Database Management System (RDBMS)
14 0.89542836 1652 high scalability-2014-05-21-9 Principles of High Performance Programs
15 0.89542603 1186 high scalability-2012-02-02-The Data-Scope Project - 6PB storage, 500GBytes-sec sequential IO, 20M IOPS, 130TFlops
16 0.89513528 72 high scalability-2007-08-22-Wikimedia architecture
18 0.89457542 498 high scalability-2009-01-20-Product: Amazon's SimpleDB
19 0.89404994 1626 high scalability-2014-04-04-Stuff The Internet Says On Scalability For April 4th, 2014
20 0.89374149 1302 high scalability-2012-08-10-Stuff The Internet Says On Scalability For August 10, 2012