high_scalability high_scalability-2009 high_scalability-2009-501 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Hello, I'm developing a product I thought of. As a part of it, I'm trying to figure out the best architecture for the product. It's a server application, what is supposed to serve A LOT of users at a time. Think of an IM application, but not web style, more like ICQ\MSN applications... With web orientation. I've read all about Meebo, Facebook Chat & etc architectures. But I'm still not sure where to start on ICQ style (this is the first phase before I go totally web). Can you direct me to some information on load balancing 10 million users? ;-) I just don't know here to begin... Thanks, Amit
sentIndex sentText sentNum sentScore
1 As a part of it, I'm trying to figure out the best architecture for the product. [sent-2, score-0.437]
2 It's a server application, what is supposed to serve A LOT of users at a time. [sent-3, score-0.478]
3 Think of an IM application, but not web style, more like ICQ\MSN applications. [sent-4, score-0.155]
4 I've read all about Meebo, Facebook Chat & etc architectures. [sent-8, score-0.183]
5 But I'm still not sure where to start on ICQ style (this is the first phase before I go totally web). [sent-9, score-0.976]
6 Can you direct me to some information on load balancing 10 million users? [sent-10, score-0.472]
wordName wordTfidf (topN-words)
[('icq', 0.643), ('msn', 0.295), ('style', 0.282), ('im', 0.225), ('supposed', 0.19), ('chat', 0.179), ('totally', 0.166), ('phase', 0.164), ('direct', 0.158), ('developing', 0.133), ('figure', 0.131), ('serve', 0.126), ('web', 0.126), ('etc', 0.12), ('users', 0.116), ('balancing', 0.114), ('thought', 0.107), ('trying', 0.106), ('sure', 0.098), ('application', 0.094), ('facebook', 0.092), ('start', 0.086), ('product', 0.083), ('million', 0.074), ('part', 0.072), ('information', 0.07), ('best', 0.07), ('read', 0.063), ('still', 0.063), ('go', 0.061), ('architecture', 0.058), ('know', 0.057), ('first', 0.056), ('load', 0.056), ('lot', 0.051), ('server', 0.046), ('like', 0.029)]
simIndex simValue blogId blogTitle
same-blog 1 1.0 501 high scalability-2009-01-25-Where do I start?
Introduction: Hello, I'm developing a product I thought of. As a part of it, I'm trying to figure out the best architecture for the product. It's a server application, what is supposed to serve A LOT of users at a time. Think of an IM application, but not web style, more like ICQ\MSN applications... With web orientation. I've read all about Meebo, Facebook Chat & etc architectures. But I'm still not sure where to start on ICQ style (this is the first phase before I go totally web). Can you direct me to some information on load balancing 10 million users? ;-) I just don't know here to begin... Thanks, Amit
Introduction: Who's Hiring? Imo is Hiring an Operations Engineer for a startup web-based service supporting multiple IM protocols. Appirio is hiring 5 core product engineers for one of Silicon's Valley's "most promising startups". Tuenti is hiring both a FrontEnd and BackEnd Engineer . Undertone is looking for a Software Engineer and a QA Engineer . Fun and Informative Events Announcing My Webinar On December 14th: What Should I Do? Choosing SQL, NoSQL Or Both For Scalable Web Applications Cool Products and Services Learn how Kabam enlisted Joyent to be the cloud environment for its growing game empire. http://www.joyent.com/customers/kabam/ CloudSigma . Instantly scalable European cloud servers. ManageEngine Applications Manager : Monitor physical, virtual and Cloud Applications . www.site24x7.com : Monitor End User Experience from a global monitoring network. Announcing December 14th Webinar: What Should I Do? Choosing SQL, NoSQL Or
3 0.12233295 318 high scalability-2008-05-14-New Facebook Chat Feature Scales to 70 Million Users Using Erlang
Introduction: Update : Erlang at Facebook by Eugene Letuchy. How Facebook uses Erlang to implement Chat, AIM Presence, and Chat Jabber support. I've done some XMPP development so when I read Facebook was making a Jabber chat client I was really curious how they would make it work. While core XMPP is straightforward, a number of protocol extensions like discovery, forms, chat states, pubsub, multi user chat, and privacy lists really up the implementation complexity. Some real engineering challenges were involved to make this puppy scale and perform. It's not clear what extensions they've implemented, but a blog entry by Facebook's Eugene Letuchy hits some of the architectural challenges they faced and how they overcame them. A web based Jabber client poses a few problems because XMPP, like most IM protocols, is an asynchronous event driven system that pretty much assumes you have a full time open connection. After logging in the server sends a client roster information and presence info
4 0.10629186 427 high scalability-2008-10-22-Server load balancing architectures, Part 2: Application-level load balancing
Introduction: The transport-level server load balancing architectures described in the first half of this article are more than adequate for many Web sites, but more complex and dynamic sites can't depend on them. Applications that rely on cache or session data must be able to handle a sequence of requests from the same client accurately and efficiently, without failing. In this follow up to his introduction to server load balancing, Gregor Roth discusses various application-level load balancing architectures, helping you decide which one will best meet the business requirements of your Web site. The first half of this article describes transport-level server load balancing solutions, such as TCP/IP-based load balancers, and analyzes their benefits and disadvantages. Load balancing on the TCP/IP level spreads incoming TCP connections over the real servers in a server farm. It is sufficient in most cases, especially for static Web sites. However, support for dynamic Web sites often re
Introduction: Who's Hiring? Imo is Hiring an Operations Engineer for a startup web-based service supporting multiple IM protocols. Appirio is hiring 5 core product engineers for one of Silicon's Valley's "most promising startups". Tuenti is hiring both a FrontEnd and BackEnd Engineer . Playfish is hiring an Engineering Director, Marketing & CRM and a Sr. Systems Operation Engineer . Electronic Arts Mobile is looking for Server Engineers . Tagged is hiring Engineers, Product Managers, Game Designers , and more. Undertone is looking for a Software Engineer and a QA Engineer . Fun and Informative Events Membase Meetups Coming to Major US Cities. The first of these technical meetups is on November 10th at Hewlett Packard in Cupertino. Cool Products and Services Learn how Kabam enlisted Joyent to be the cloud environment for its growing game empire. http://www.joyent.com/customers/kabam/ CloudSigma . Instantly scalable European cloud servers.
6 0.096625984 840 high scalability-2010-06-10-The Four Meta Secrets of Scaling at Facebook
8 0.084884748 569 high scalability-2009-04-14-Scalability resources
9 0.082725123 244 high scalability-2008-02-11-Yahoo Live's Scaling Problems Prove: Release Early and Often - Just Don't Screw Up
10 0.078730032 21 high scalability-2007-07-23-GoogleTalk Architecture
12 0.076768018 1123 high scalability-2011-09-23-The Real News is Not that Facebook Serves Up 1 Trillion Pages a Month…
13 0.076310448 465 high scalability-2008-12-14-Scaling MySQL on a 256-way T5440 server using Solaris ZFS and Java 1.7
14 0.075564906 720 high scalability-2009-10-12-High Performance at Massive Scale – Lessons learned at Facebook
15 0.070845731 1573 high scalability-2014-01-06-How HipChat Stores and Indexes Billions of Messages Using ElasticSearch and Redis
16 0.068977863 379 high scalability-2008-09-04-Database question for upcoming project
17 0.066865399 1444 high scalability-2013-04-23-Facebook Secrets of Web Performance
18 0.066146307 313 high scalability-2008-05-02-Friends for Sale Architecture - A 300 Million Page View-Month Facebook RoR App
19 0.064866215 1359 high scalability-2012-11-15-Gone Fishin': Justin.Tv's Live Video Broadcasting Architecture
20 0.064668149 796 high scalability-2010-03-16-Justin.tv's Live Video Broadcasting Architecture
topicId topicWeight
[(0, 0.105), (1, 0.026), (2, 0.018), (3, -0.078), (4, 0.021), (5, -0.063), (6, -0.044), (7, 0.001), (8, 0.001), (9, 0.025), (10, 0.013), (11, 0.054), (12, -0.021), (13, -0.002), (14, -0.022), (15, -0.001), (16, 0.03), (17, 0.015), (18, 0.032), (19, 0.024), (20, 0.049), (21, 0.015), (22, 0.007), (23, -0.044), (24, 0.06), (25, -0.042), (26, 0.031), (27, -0.018), (28, 0.009), (29, 0.006), (30, -0.031), (31, -0.007), (32, 0.023), (33, -0.056), (34, 0.025), (35, -0.004), (36, -0.042), (37, -0.022), (38, 0.005), (39, -0.057), (40, -0.033), (41, 0.018), (42, 0.032), (43, -0.015), (44, 0.012), (45, 0.032), (46, -0.002), (47, 0.023), (48, 0.018), (49, -0.015)]
simIndex simValue blogId blogTitle
same-blog 1 0.94428456 501 high scalability-2009-01-25-Where do I start?
Introduction: Hello, I'm developing a product I thought of. As a part of it, I'm trying to figure out the best architecture for the product. It's a server application, what is supposed to serve A LOT of users at a time. Think of an IM application, but not web style, more like ICQ\MSN applications... With web orientation. I've read all about Meebo, Facebook Chat & etc architectures. But I'm still not sure where to start on ICQ style (this is the first phase before I go totally web). Can you direct me to some information on load balancing 10 million users? ;-) I just don't know here to begin... Thanks, Amit
2 0.71901649 129 high scalability-2007-10-23-Hire Facebook, Ning, and Salesforce to Scale for You
Introduction: One of the premier scaling strategies is always: get someone else to do the work for you. But unlike Huckleberry Finn in Tom Sawyer , you won't have to trick anyone into whitewashing a fence for you. Times have changed. Companies like Ning , Facebook , and Salesforce are more than happy to help. Their price: lock-in. Previously you had few options when building a "real" website. You needed to do everything yourself. Infrastructure and application were all yours. Then companies stepped in by commoditizing parts of the infrastructure, but the application was still yours. The next step is full on Borg take no prisoners assimilation where the infrastructure and application are built as one collective. What you have to decide as someone faced with building a scalable website is if these new options are worth the price. Feeding this explosion of choice is one of the new strategy games on the intertubes: the Internet Platform Game. Ning's Marc Andreessen defines a platform as: a
3 0.70851254 1123 high scalability-2011-09-23-The Real News is Not that Facebook Serves Up 1 Trillion Pages a Month…
Introduction: There’s some amount of debate whether Facebook really crossed over the one trillion page view per month threshold. While one report says it did, another respected firm says it did not ; that its monthly page views are a mere 467 billion per month. In the big scheme of things, the discrepancy is somewhat irrelevant, as neither show the true load on Facebook’s infrastructure – which is far more impressive a set of numbers than its externally measured “page view” metric. Mashable reported in “ Facebook Surpasses 1 Trillion Pageviews per Month ” that the social networking giant saw “approximately 870 million unique visitors in June and 860 million in July” and followed up with some per visitor statistics, indicating “each visitor averaged approximately 1,160 page views in July and 40 per visit — enormous by any standard. Time spent on the site was around 25 minutes per user.” From an architectural standpoint it’s not just about the page views. It’s about requests and responses,
4 0.69262505 509 high scalability-2009-02-05-Product: HAProxy - The Reliable, High Performance TCP-HTTP Load Balancer
Introduction: Update: Load Balancing in Amazon EC2 with HAProxy. Grig Gheorghiu writes a nice post on HAProxy functionality and configuration: Emulating virtual servers, Logging, SSL, Load balancing algorithms, Session persistence with cookies, Server health checks, etc. Adapted From the website: HAProxy is a free, very fast and reliable solution offering high availability, load balancing, and proxying for TCP and HTTP-based applications. It is particularly suited for web sites crawling under very high loads while needing persistence or Layer7 processing. Supporting tens of thousands of connections is clearly realistic with todays hardware. Its mode of operation makes its integration into existing architectures very easy and riskless, while still offering the possibility not to expose fragile web servers to the Net. Currently, two major versions are supported : * version 1.1 - maintains critical sites online since 200 The most stable and reliable, has reached years of uptime. Receive
5 0.66732281 840 high scalability-2010-06-10-The Four Meta Secrets of Scaling at Facebook
Introduction: Aditya Agarwal, Director of Engineering at Facebook, gave an excellent Scale at Facebook talk that covers their architecture, but the talk is really more about how to scale an organization by preserving the best parts of its culture. The key take home of the talk is: You can get the code right, you can get the products right, but you need to get the culture right first. If you don't get the culture right then your company won't scale. This leads into the four meta secrets of scaling at Facebook: Scaling takes Iteration Don't Over Design Choose the right tool for the job, but realize that your choice comes with overhead. Get the culture right. Move Fast - break things. Huge Impact - small teams. Be bold - innovate. Some Background Facebook is big : 400 million active users; users spend an average of 20 minutes a day; 5 billion pieces of content (status updates, comments, likes, photo uploads, video uploads, chat messages, inbox messages, group events, f
6 0.65780234 700 high scalability-2009-09-10-The technology behind Tornado, FriendFeed's web server
7 0.64789939 356 high scalability-2008-07-22-Scaling Bumper Sticker: A 1 Billion Page Per Month Facebook RoR App
8 0.63876778 1602 high scalability-2014-02-26-The WhatsApp Architecture Facebook Bought For $19 Billion
9 0.63665593 620 high scalability-2009-06-05-SSL RPC API Scalability
10 0.63535339 771 high scalability-2010-02-04-Hot Scalability Links for February 4, 2010
11 0.62975353 318 high scalability-2008-05-14-New Facebook Chat Feature Scales to 70 Million Users Using Erlang
12 0.62834954 870 high scalability-2010-08-02-7 Scaling Strategies Facebook Used to Grow to 500 Million Users
13 0.62438619 313 high scalability-2008-05-02-Friends for Sale Architecture - A 300 Million Page View-Month Facebook RoR App
14 0.62415421 792 high scalability-2010-03-10-How FarmVille Scales - The Follow-up
15 0.61910892 1323 high scalability-2012-09-15-4 Reasons Facebook Dumped HTML5 and Went Native
16 0.61518967 427 high scalability-2008-10-22-Server load balancing architectures, Part 2: Application-level load balancing
17 0.61505789 1444 high scalability-2013-04-23-Facebook Secrets of Web Performance
18 0.61055475 569 high scalability-2009-04-14-Scalability resources
19 0.60468006 774 high scalability-2010-02-08-How FarmVille Scales to Harvest 75 Million Players a Month
20 0.60365659 426 high scalability-2008-10-22-Server load balancing architectures, Part 1: Transport-level load balancing
topicId topicWeight
[(1, 0.072), (2, 0.217), (36, 0.22), (61, 0.321)]
simIndex simValue blogId blogTitle
same-blog 1 0.90234488 501 high scalability-2009-01-25-Where do I start?
Introduction: Hello, I'm developing a product I thought of. As a part of it, I'm trying to figure out the best architecture for the product. It's a server application, what is supposed to serve A LOT of users at a time. Think of an IM application, but not web style, more like ICQ\MSN applications... With web orientation. I've read all about Meebo, Facebook Chat & etc architectures. But I'm still not sure where to start on ICQ style (this is the first phase before I go totally web). Can you direct me to some information on load balancing 10 million users? ;-) I just don't know here to begin... Thanks, Amit
2 0.86839855 322 high scalability-2008-05-19-Conference: Infoscale 2008 in Italy (June 4-6)
Introduction: The Third International Conference on Scalable Information Systems will focus on a wide array of scalability issues and investigate new approaches to tackle problems arising from the ever-growing size and complexity of information of all kinds. Looking at their technical program a lot of interesting topics will be covered. I see sensor networks, a subject I'm really interested in, has a number of sessions. That's unusual. And it's in Italy!
3 0.86698616 208 high scalability-2008-01-11-FTP Sanity: Redundancy, archiving, consolidation.
Introduction: Easy FTP redundancy and consolidation with the Open Source project Generic-FTP. Works with probably any Linux FTP Server (ProFTPD only one tested). Get rid of some single points of failure. A very easy to set up solution using scripts written in PHP. Tested thoroughly in a production environment.
4 0.86315322 173 high scalability-2007-12-05-Easier Production Releases
Introduction: I’ve been a part of some late night release procedures and they’re never fun. You’ve got QA, Dev, IT and a handful of managers sitting in their jammies in a group IM (or worse, a conference call) from 2:00 AM until way too early in the morning. Everyone’s grumpy and sleepy, causing the release to be more difficult and take longer. Sometimes the dreaded “rollback!” is yelled. All this because you’re running a high profile website that needs to be accessible 24/7, and 2:00 AM - 5:00 AM downtime is better than daytime downtime. If you're a site that doesn't have 10s of thousands to drop on a real http load balancer, use this strategy to release software during business hours with no downtime using apache's mod_proxy_balancer....
5 0.85989147 930 high scalability-2010-10-28-NoSQL Took Away the Relational Model and Gave Nothing Back
Introduction: Update : Benjamin Black said he was the source of the quote and also said I was wrong about what he meant. His real point: The meaning of the statement was that NoSQL systems (really the various map-reduce systems) are lacking a standard model for describing and querying and that developing one should be a high priority task for them. At the A NoSQL Evening in Palo Alto , an audience member, sorry, I couldn't tell who, said something I found really interesting: NoSQL took away the relational model and gave nothing back. The idea being that NoSQL has focussed on ease of use, scalability, performance, etc, but it has lost the idea of how data relates to other data. True to its name, the relational model is very good at capturing a managing relationships. With NoSQL all relationships have been pushed back onto the poor programmer to implement in code rather than the database managing it. We've sacrificed usability. NoSQL is about concurrency, latency, and scalability, but it
6 0.85913324 675 high scalability-2009-08-08-1dbase vs. many and cloud hosting vs. dedicated server(s)?
7 0.85028899 1287 high scalability-2012-07-20-Stuff The Internet Says On Scalability For July 20, 2012
8 0.84533441 793 high scalability-2010-03-10-Saying Yes to NoSQL; Going Steady with Cassandra at Digg
9 0.84501565 347 high scalability-2008-07-07-Five Ways to Stop Framework Fixation from Crashing Your Scaling Strategy
10 0.84449279 493 high scalability-2009-01-16-Just-In-Time Scalability: Agile Methods to Support Massive Growth (IMVU case study)
11 0.84301865 332 high scalability-2008-05-28-Job queue and search engine
12 0.83333302 1303 high scalability-2012-08-13-Ask HighScalability: Facing scaling issues with news feeds on Redis. Any advice?
13 0.83059561 1254 high scalability-2012-05-30-Strategy: Get Servers for Free and Make Users Happy by Turning on Compression
14 0.82894057 1184 high scalability-2012-01-31-Performance in the Cloud: Business Jitter is Bad
15 0.827757 99 high scalability-2007-09-23-HA for switches
16 0.82730091 1411 high scalability-2013-02-22-Stuff The Internet Says On Scalability For February 22, 2013
17 0.81748092 150 high scalability-2007-11-12-Slashdot Architecture - How the Old Man of the Internet Learned to Scale
18 0.81703126 580 high scalability-2009-04-24-INFOSCALE 2009 in June in Hong Kong
19 0.81350249 549 high scalability-2009-03-26-Performance - When do I start worrying?
20 0.81115496 1201 high scalability-2012-02-29-Strategy: Put Mobile Video Into Cold Storage After 30 Days