high_scalability high_scalability-2009 high_scalability-2009-550 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Ebay [1] Starts in 1995, initial name AuctionWeb (V1) : - very simple architecture - based on perl - no database, for data persistence they used plain files Because of rapid growth they needed to improve their architecture and so V2 (clever name) was born: - replaced perl with C/C++ - started using a database in a master-slave configuration - C++ back-end - XSLT front-end Any request will lead to an XML file being created in C++ and the XLST processor will transform that into html. *pretty sophisticated architecture for the 90s, XLST was cutting-edge back then* That hold out pretty well for a while but in the late 90s ebay experienced an exponential growth. They started having some trouble with outages and needed improvements, so V3 was developed: - based on java - search engine still used C++ - proof that relational databases can scale (aggressive caching) - developed a messaging layer for making a lot of asyncronious calls, they a
sentIndex sentText sentNum sentScore
1 *pretty sophisticated architecture for the 90s, XLST was cutting-edge back then* That hold out pretty well for a while but in the late 90s ebay experienced an exponential growth. [sent-2, score-0.869]
2 For more details check: Eclipse at Ebay Tailoring Eclipse to the eBay architecture [1] Images and ideas/info are from the links above. [sent-5, score-0.083]
wordName wordTfidf (topN-words)
[('xlst', 0.359), ('eclipse', 0.328), ('ebay', 0.324), ('java', 0.245), ('xml', 0.17), ('replaced', 0.169), ('xslt', 0.163), ('started', 0.158), ('boom', 0.14), ('images', 0.131), ('name', 0.123), ('developed', 0.122), ('born', 0.112), ('loves', 0.111), ('aggressive', 0.106), ('plain', 0.103), ('pretty', 0.102), ('exponential', 0.101), ('posted', 0.1), ('outages', 0.099), ('request', 0.098), ('principle', 0.098), ('trouble', 0.097), ('transform', 0.097), ('switched', 0.096), ('delay', 0.093), ('clever', 0.093), ('doubt', 0.092), ('generating', 0.091), ('item', 0.091), ('ended', 0.091), ('late', 0.091), ('needed', 0.09), ('ajax', 0.089), ('proof', 0.089), ('appear', 0.089), ('hold', 0.084), ('classes', 0.084), ('sophisticated', 0.084), ('architecture', 0.083), ('persistence', 0.082), ('although', 0.079), ('became', 0.076), ('improvements', 0.076), ('perl', 0.075), ('initial', 0.075), ('flash', 0.075), ('processor', 0.072), ('remove', 0.069), ('link', 0.068)]
simIndex simValue blogId blogTitle
same-blog 1 1.0 550 high scalability-2009-03-30-Ebay history and architecture
Introduction: Ebay [1] Starts in 1995, initial name AuctionWeb (V1) : - very simple architecture - based on perl - no database, for data persistence they used plain files Because of rapid growth they needed to improve their architecture and so V2 (clever name) was born: - replaced perl with C/C++ - started using a database in a master-slave configuration - C++ back-end - XSLT front-end Any request will lead to an XML file being created in C++ and the XLST processor will transform that into html. *pretty sophisticated architecture for the 90s, XLST was cutting-edge back then* That hold out pretty well for a while but in the late 90s ebay experienced an exponential growth. They started having some trouble with outages and needed improvements, so V3 was developed: - based on java - search engine still used C++ - proof that relational databases can scale (aggressive caching) - developed a messaging layer for making a lot of asyncronious calls, they a
2 0.20013702 331 high scalability-2008-05-27-eBay Architecture
Introduction: Update 2: EBay's Randy Shoup spills the secrets of how to service hundreds of millions of users and over two billion page views a day in Scalability Best Practices: Lessons from eBay on InfoQ. The practices: Partition by Function, Split Horizontally, Avoid Distributed Transactions, Decouple Functions Asynchronously, Move Processing To Asynchronous Flows, Virtualize At All Levels, Cache Appropriately. Update: eBay Serves 5 Billion API Calls Each Month . Aren't we seeing more and more traffic driven by mashups composed on top of open APIs? APIs are no longer a bolt on, they are your application. Architecturally that argues for implementing your own application around the same APIs developers and users employ. Who hasn't wondered how eBay does their business? As one of the largest most loaded websites in the world, it can't be easy. And the subtitle of the presentation hints at how creating such a monster system requires true engineering: Striking a balance between site stabilit
3 0.1692767 459 high scalability-2008-12-03-Java World Interview on Scalability and Other Java Scalability Secrets
Introduction: OK, this interview is with me on Java scalability issues. I sound like a bigger idiot than I would like, but I suppose it could have been worse. The Java World folks were very nice and did a good job, so there’s no blame on them :-) The interview went an interesting direction, but there’s more I’d like add and I will do so here. Two major rules regarding Java and scalability have popped out at me: Java – It’s the platform stupid . Java the language isn’t the big win. What is the big win is the ecosystem building up around the JVM, libraries, and toolsets. Java – It’s the community stupid . A lot of creativity is being expended on leveraging the Java platform to meet scalability challenges. The amazing community that has built up around Java is pushing Java to the next level in almost every direction imaginable. The fecundity of the Java ecosystem can most readily be seen with the efforts to tame our multi-core future. There’s a multi-core crisis going in case you haven’t
4 0.11656207 113 high scalability-2007-10-07-Paper: Architecture of a Highly Scalable NIO-Based Server
Introduction: The article describes the basic architecture of a connection-oriented NIO-based java server. It takes a look at a preferred threading model, Java Non-blocking I/O and discusses the basic components of such a server.
5 0.11243584 742 high scalability-2009-11-17-10 eBay Secrets for Planet Wide Scaling
Introduction: You don't even have to make a bid, Randy Shoup, an eBay Distinguished Architect, gives this presentation on how eBay scales, for free. Randy has done a fabulous job in this presentation and in other talks listed at the end of this post getting at the heart of the principles behind scalability. It's more about ideas of how things work and fit together than a focusing on a particular technology stack. Impressive Stats In case you weren't sure, eBay is big, with lots of: users, data, features, and change... Over 89 million active users worldwide 190 million items for sale in 50,000 categories Over 8 billion URL requests per day Hundreds of new features per quarter Roughly 10% of items are listed or ended every day In 39 countries and 10 languages 24x7x365 70 billion read / write operations / day Processes 50TB of new, incremental data per day Analyzes 50PB of data per day 10 Lessons The presentation does a good job explaining each lesson, but the list is.
7 0.097224966 682 high scalability-2009-08-16-ThePort Network Architecture
8 0.093752481 1118 high scalability-2011-09-19-Big Iron Returns with BigMemory
9 0.09063217 320 high scalability-2008-05-17-DB2 Express-C
10 0.089510344 471 high scalability-2008-12-19-Gigaspaces curbs latency outliers with Java Real Time
11 0.085625276 5 high scalability-2007-07-10-mixi.jp Architecture
12 0.083983354 425 high scalability-2008-10-22-Scalability Best Practices: Lessons from eBay
13 0.082997948 291 high scalability-2008-03-29-20 New Rules for Faster Web Pages
14 0.079012126 94 high scalability-2007-09-17-Blog: Adding Simplicity by Dan Pritchett
15 0.078126565 121 high scalability-2007-10-14-Newbie in scalability design issues
16 0.077955559 1440 high scalability-2013-04-15-Scaling Pinterest - From 0 to 10s of Billions of Page Views a Month in Two Years
17 0.077943146 1158 high scalability-2011-12-16-Stuff The Internet Says On Scalability For December 16, 2011
18 0.074337199 124 high scalability-2007-10-16-How Scalable are Single Page Ajax Apps?
19 0.074083418 1024 high scalability-2011-04-15-Stuff The Internet Says On Scalability For April 15, 2011
20 0.073961772 538 high scalability-2009-03-16-Are Cloud Based Memory Architectures the Next Big Thing?
topicId topicWeight
[(0, 0.135), (1, 0.04), (2, -0.037), (3, -0.048), (4, 0.04), (5, -0.008), (6, 0.008), (7, 0.011), (8, -0.004), (9, 0.064), (10, -0.033), (11, -0.009), (12, 0.007), (13, -0.009), (14, 0.005), (15, -0.055), (16, -0.023), (17, -0.039), (18, 0.027), (19, -0.048), (20, -0.026), (21, -0.044), (22, -0.002), (23, -0.026), (24, -0.02), (25, -0.013), (26, -0.01), (27, -0.07), (28, 0.036), (29, 0.006), (30, -0.019), (31, 0.042), (32, -0.024), (33, -0.059), (34, 0.033), (35, -0.002), (36, -0.014), (37, -0.016), (38, -0.018), (39, -0.008), (40, -0.002), (41, -0.031), (42, 0.0), (43, 0.091), (44, -0.016), (45, 0.007), (46, 0.13), (47, 0.021), (48, -0.088), (49, 0.044)]
simIndex simValue blogId blogTitle
same-blog 1 0.97779167 550 high scalability-2009-03-30-Ebay history and architecture
Introduction: Ebay [1] Starts in 1995, initial name AuctionWeb (V1) : - very simple architecture - based on perl - no database, for data persistence they used plain files Because of rapid growth they needed to improve their architecture and so V2 (clever name) was born: - replaced perl with C/C++ - started using a database in a master-slave configuration - C++ back-end - XSLT front-end Any request will lead to an XML file being created in C++ and the XLST processor will transform that into html. *pretty sophisticated architecture for the 90s, XLST was cutting-edge back then* That hold out pretty well for a while but in the late 90s ebay experienced an exponential growth. They started having some trouble with outages and needed improvements, so V3 was developed: - based on java - search engine still used C++ - proof that relational databases can scale (aggressive caching) - developed a messaging layer for making a lot of asyncronious calls, they a
Introduction: PayPal gives yet another glowing report of an app rewritten in node.js experiencing substantial performance improvements. PayPal rewrote their account overview page, one of the most trafficked apps on the website, which was previously written in King Java. The benefits: Full-stack engineers. Using JavaScript on both the front-end and the back-end removed an artificial boundary between the browser and server, allowing engineers to code both. Built almost twice as fast with fewer people Written in 33% fewer lines of code Constructed with 40% fewer files Double the requests per second vs. the Java application. 35% decrease in the average response time for the same page. A common pro Java response is an argument like clearly these people don't know how to program Java. Or rewriting an application usually makes it faster. Or the benchmark is faulty. And so on. Consider it noted. These are all potential factors. Baron Schwartz from VividCortex has a different tak
3 0.72774631 459 high scalability-2008-12-03-Java World Interview on Scalability and Other Java Scalability Secrets
Introduction: OK, this interview is with me on Java scalability issues. I sound like a bigger idiot than I would like, but I suppose it could have been worse. The Java World folks were very nice and did a good job, so there’s no blame on them :-) The interview went an interesting direction, but there’s more I’d like add and I will do so here. Two major rules regarding Java and scalability have popped out at me: Java – It’s the platform stupid . Java the language isn’t the big win. What is the big win is the ecosystem building up around the JVM, libraries, and toolsets. Java – It’s the community stupid . A lot of creativity is being expended on leveraging the Java platform to meet scalability challenges. The amazing community that has built up around Java is pushing Java to the next level in almost every direction imaginable. The fecundity of the Java ecosystem can most readily be seen with the efforts to tame our multi-core future. There’s a multi-core crisis going in case you haven’t
4 0.69784445 113 high scalability-2007-10-07-Paper: Architecture of a Highly Scalable NIO-Based Server
Introduction: The article describes the basic architecture of a connection-oriented NIO-based java server. It takes a look at a preferred threading model, Java Non-blocking I/O and discusses the basic components of such a server.
5 0.6678071 331 high scalability-2008-05-27-eBay Architecture
Introduction: Update 2: EBay's Randy Shoup spills the secrets of how to service hundreds of millions of users and over two billion page views a day in Scalability Best Practices: Lessons from eBay on InfoQ. The practices: Partition by Function, Split Horizontally, Avoid Distributed Transactions, Decouple Functions Asynchronously, Move Processing To Asynchronous Flows, Virtualize At All Levels, Cache Appropriately. Update: eBay Serves 5 Billion API Calls Each Month . Aren't we seeing more and more traffic driven by mashups composed on top of open APIs? APIs are no longer a bolt on, they are your application. Architecturally that argues for implementing your own application around the same APIs developers and users employ. Who hasn't wondered how eBay does their business? As one of the largest most loaded websites in the world, it can't be easy. And the subtitle of the presentation hints at how creating such a monster system requires true engineering: Striking a balance between site stabilit
6 0.65786344 364 high scalability-2008-08-14-Product: Terracotta - Open Source Network-Attached Memory
7 0.65000373 82 high scalability-2007-09-06-Why doesn't anyone use j2ee?
8 0.63919771 471 high scalability-2008-12-19-Gigaspaces curbs latency outliers with Java Real Time
9 0.62274635 145 high scalability-2007-11-08-ID generator
10 0.61103415 235 high scalability-2008-02-02-The case against ORM Frameworks in High Scalability Architectures
11 0.60802567 124 high scalability-2007-10-16-How Scalable are Single Page Ajax Apps?
12 0.60643125 384 high scalability-2008-09-16-EE-Appserver Clustering OR Terracota OR Coherence OR something else?
13 0.60213941 320 high scalability-2008-05-17-DB2 Express-C
14 0.58085507 440 high scalability-2008-11-11-Arhcitecture for content management
15 0.57075548 135 high scalability-2007-10-27-.Net2 and AJAX scalability?
16 0.56797397 1068 high scalability-2011-06-27-TripAdvisor Architecture - 40M Visitors, 200M Dynamic Page Views, 30TB Data
17 0.56726408 344 high scalability-2008-06-09-FaceStat's Rousing Tale of Scaling Woe and Wisdom Won
18 0.55636758 993 high scalability-2011-02-22-Is Node.js Becoming a Part of the Stack? SimpleGeo Says Yes.
19 0.55539799 1336 high scalability-2012-10-09-Batoo JPA - The new JPA Implementation that runs over 15 times faster...
20 0.55446231 800 high scalability-2010-03-26-Strategy: Caching 404s Saved the Onion 66% on Server Time
topicId topicWeight
[(1, 0.162), (2, 0.23), (30, 0.052), (47, 0.246), (61, 0.111), (79, 0.073), (85, 0.025)]
simIndex simValue blogId blogTitle
1 0.96925145 57 high scalability-2007-08-03-Scaling IMAP and POP3
Introduction: Just thought I'd drop a brief suggestion to anyone building a large mail system. Our solution for scaling mail pickup was to develop a sharded architecture whereby accounts are spread across a cluster of servers, each with imap/pop3 capability. Then we use a cluster of reverse proxies (Perdition) speaking to the backend imap/pop3 servers . The benefit of this approach is you can use simply use round-robin or HA loadbalancing on the perdition servers that end users connect to (e.g. admins can easily move accounts around on the backend storage servers without affecting end users). Perdition manages routing users to the appropriate backend servers and has MySQL support. What we also liked about this approach was that it had no dependency on a distributed or networked filesystem, so less chance of corruption or data consistency issues. When an individual server reaches capacity, we just off load users to a less used server. If any server goes offline, it only affects the fraction of users
2 0.96559972 81 high scalability-2007-09-06-Scaling IMAP and POP3
Introduction: Another scalability strategy brought to you by Erik Osterman: Just thought I'd drop a brief suggestion to anyone building a large mail system. Our solution for scaling mail pickup was to develop a sharded architecture whereby accounts are spread across a cluster of servers, each with imap/pop3 capability. Then we use a cluster of reverse proxies (Perdition) speaking to the backend imap/pop3 servers . The benefit of this approach is you can use simply use round-robin or HA load balancing on the perdition servers that end users connect to (e.g. admins can easily move accounts around on the backend storage servers without affecting end users). Perdition manages routing users to the appropriate backend servers and has MySQL support. What we also liked about this approach was that it had no dependency on a distributed or networked file system, so less chance of corruption or data consistency issues. When an individual server reaches capacity, we just off load users to a less u
3 0.94817662 756 high scalability-2009-12-30-Terrastore - Scalable, elastic, consistent document store.
Introduction: Terrastore is a new-born document store which provides advanced scalability and elasticity features without sacrificing consistency. Here are a few highlights: Ubiquitous: based on the universally supported HTTP protocol. Distributed: nodes can run and live everywhere on your network. Elastic: you can add and remove nodes dynamically to/from your running cluster with no downtime and no changes at all to your configuration. Scalable at the data layer: documents are partitioned and distributed among your nodes, with automatic and transparent re-balancing when nodes join and leave. Scalable at the computational layer: query and update operations are distributed to the nodes which actually holds the queried/updated data, minimizing network traffic and spreading computational load. Consistent: providing per-document consistency, you're guaranteed to always get the latest value of a single document, with read committed isolation for concurrent modifications. Schemales
4 0.94279289 163 high scalability-2007-11-21-n-phase commit for FS writes, reads stay local
Introduction: I am try i ng to f i nd a L i nux FS that wi l l a l low me to rep l icate a l l wr i tes synchronous l y to n nodes i n a web server c l uster, wh i le keep i ng a l l reads local. It shou l d not require specialized hardware.
5 0.9391371 760 high scalability-2010-01-13-10 Hot Scalability Links for January 13, 2010
Introduction: Has Amazon EC2 become over subscribed? by Alan Williamson. Systemic problems hit AWS as users experience problems across Amazon's infrastructure. It seems the strange attractor of a cloud may be the same as for a shared hosting service. Understanding Infrastructure 2.0 by James Urquhart. We need to take a systems view of our entire infrastructure, and build our automation around the end-to-end architecture of that system . Hey You, Get Off of My Cloud: Exploring Information Leakage in Third-Party Compute Clouds . We show that it is possible to map the internal cloud infrastructure . Hadoop World: Building Data Intensive Apps with Hadoop and EC2 by Pete Skomoroch. Dives into detail about how he built TrendingTopics.org using Hadoop and EC2 . A Crash Course in Modern Hardware by Cliff Click. Yes, your mind will hurt after watching this. And no, you probably don't know what your microprocessor is doing anymore. EVE Scalability Explained by James Harrison. This pos
6 0.93063748 852 high scalability-2010-07-07-Strategy: Recompute Instead of Remember Big Data
7 0.922297 94 high scalability-2007-09-17-Blog: Adding Simplicity by Dan Pritchett
8 0.92195523 676 high scalability-2009-08-08-Yahoo!'s PNUTS Database: Too Hot, Too Cold or Just Right?
same-blog 9 0.91726041 550 high scalability-2009-03-30-Ebay history and architecture
10 0.91668576 1166 high scalability-2011-12-30-Stuff The Internet Says On Scalability For December 30, 2011
11 0.90962428 144 high scalability-2007-11-07-What CDN would you recommend?
12 0.89374542 1062 high scalability-2011-06-15-101 Questions to Ask When Considering a NoSQL Database
13 0.88684839 1326 high scalability-2012-09-20-How Vimeo Saves 50% on EC2 by Playing a Smarter Game
14 0.88404584 708 high scalability-2009-09-17-Infinispan narrows the gap between open source and commercial data caches
15 0.87787586 1530 high scalability-2013-10-11-Stuff The Internet Says On Scalability For October 11th, 2013
16 0.8749125 1054 high scalability-2011-06-06-NoSQL Pain? Learn How to Read-write Scale Without a Complete Re-write
17 0.86080492 1440 high scalability-2013-04-15-Scaling Pinterest - From 0 to 10s of Billions of Page Views a Month in Two Years
18 0.85686862 276 high scalability-2008-03-15-New Website Design Considerations
19 0.8327921 358 high scalability-2008-07-26-Sharding the Hibernate Way
20 0.82662016 986 high scalability-2011-02-10-Database Isolation Levels And Their Effects on Performance and Scalability