high_scalability high_scalability-2007 high_scalability-2007-80 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Perdition is a fully featured POP3 and IMAP4 proxy server. It is able to handle both SSL and non-SSL connections and redirect users to a real-server based on a database lookup. Perdition supports modular based database access. ODBC, MySQL, PostgreSQL, GDBM, POSIX Regular Expression and NIS modules ship with the distribution. The API for modules is open allowing arbitrary modules to be written to allow access to any data store. Perdition has many uses. Including, creating large mail systems where an end-user's mailbox may be stored on one of several hosts, integrating different mail systems together, migrating between different email infrastructures, and bridging plain-text, SSL and TLS services. It can also be used as part of a firewall. The use of perditon to scale mail services beyond a single box is discussed in high capacity email.
sentIndex sentText sentNum sentScore
1 Perdition is a fully featured POP3 and IMAP4 proxy server. [sent-1, score-0.337]
2 It is able to handle both SSL and non-SSL connections and redirect users to a real-server based on a database lookup. [sent-2, score-0.49]
3 ODBC, MySQL, PostgreSQL, GDBM, POSIX Regular Expression and NIS modules ship with the distribution. [sent-4, score-0.488]
4 The API for modules is open allowing arbitrary modules to be written to allow access to any data store. [sent-5, score-1.136]
5 Including, creating large mail systems where an end-user's mailbox may be stored on one of several hosts, integrating different mail systems together, migrating between different email infrastructures, and bridging plain-text, SSL and TLS services. [sent-7, score-2.15]
6 The use of perditon to scale mail services beyond a single box is discussed in high capacity email. [sent-9, score-0.825]
wordName wordTfidf (topN-words)
[('mail', 0.417), ('modules', 0.345), ('perdition', 0.217), ('bridging', 0.217), ('odbc', 0.217), ('posix', 0.199), ('ssl', 0.199), ('tls', 0.193), ('redirect', 0.176), ('email', 0.172), ('expression', 0.169), ('mailbox', 0.169), ('featured', 0.153), ('integrating', 0.148), ('migrating', 0.144), ('ship', 0.143), ('modular', 0.14), ('infrastructures', 0.133), ('discussed', 0.13), ('arbitrary', 0.126), ('hosts', 0.119), ('postgresql', 0.119), ('proxy', 0.112), ('beyond', 0.094), ('allowing', 0.092), ('regular', 0.089), ('supports', 0.083), ('connections', 0.081), ('box', 0.079), ('based', 0.077), ('fully', 0.072), ('allow', 0.071), ('different', 0.068), ('stored', 0.068), ('including', 0.067), ('creating', 0.067), ('systems', 0.066), ('api', 0.064), ('several', 0.063), ('written', 0.062), ('together', 0.061), ('capacity', 0.059), ('database', 0.056), ('mysql', 0.05), ('handle', 0.05), ('able', 0.05), ('access', 0.05), ('part', 0.049), ('services', 0.046), ('open', 0.045)]
simIndex simValue blogId blogTitle
same-blog 1 1.0 80 high scalability-2007-09-06-Product: Perdition Mail Retrieval Proxy
Introduction: Perdition is a fully featured POP3 and IMAP4 proxy server. It is able to handle both SSL and non-SSL connections and redirect users to a real-server based on a database lookup. Perdition supports modular based database access. ODBC, MySQL, PostgreSQL, GDBM, POSIX Regular Expression and NIS modules ship with the distribution. The API for modules is open allowing arbitrary modules to be written to allow access to any data store. Perdition has many uses. Including, creating large mail systems where an end-user's mailbox may be stored on one of several hosts, integrating different mail systems together, migrating between different email infrastructures, and bridging plain-text, SSL and TLS services. It can also be used as part of a firewall. The use of perditon to scale mail services beyond a single box is discussed in high capacity email.
2 0.22882576 57 high scalability-2007-08-03-Scaling IMAP and POP3
Introduction: Just thought I'd drop a brief suggestion to anyone building a large mail system. Our solution for scaling mail pickup was to develop a sharded architecture whereby accounts are spread across a cluster of servers, each with imap/pop3 capability. Then we use a cluster of reverse proxies (Perdition) speaking to the backend imap/pop3 servers . The benefit of this approach is you can use simply use round-robin or HA loadbalancing on the perdition servers that end users connect to (e.g. admins can easily move accounts around on the backend storage servers without affecting end users). Perdition manages routing users to the appropriate backend servers and has MySQL support. What we also liked about this approach was that it had no dependency on a distributed or networked filesystem, so less chance of corruption or data consistency issues. When an individual server reaches capacity, we just off load users to a less used server. If any server goes offline, it only affects the fraction of users
3 0.22845024 81 high scalability-2007-09-06-Scaling IMAP and POP3
Introduction: Another scalability strategy brought to you by Erik Osterman: Just thought I'd drop a brief suggestion to anyone building a large mail system. Our solution for scaling mail pickup was to develop a sharded architecture whereby accounts are spread across a cluster of servers, each with imap/pop3 capability. Then we use a cluster of reverse proxies (Perdition) speaking to the backend imap/pop3 servers . The benefit of this approach is you can use simply use round-robin or HA load balancing on the perdition servers that end users connect to (e.g. admins can easily move accounts around on the backend storage servers without affecting end users). Perdition manages routing users to the appropriate backend servers and has MySQL support. What we also liked about this approach was that it had no dependency on a distributed or networked file system, so less chance of corruption or data consistency issues. When an individual server reaches capacity, we just off load users to a less u
4 0.19738308 1501 high scalability-2013-08-13-In Memoriam: Lavabit Architecture - Creating a Scalable Email Service
Introduction: With Lavabit shutting down under murky circumstances , it seems fitting to repost an old (2009), yet still very good post by Ladar Levison on Lavabit's architecture. I don't know how much of this information is still current, but it should give you a general idea what Lavabit was all about. Getting to Know You What is the name of your system and where can we find out more about it? Note: these links are no longer valid... Lavabit http://lavabit.com http://lavabit.com/network.html http://lavabit.com/about.html What is your system for? Lavabit is a mid-sized email service provider. We currently have about 140,000 registered users with more than 260,000 email addresses. While most of our accounts belong to individual users, we also provide corporate email services to approximately 70 companies. Why did you decide to build this system? We built the system to compete against the other large free email providers, with an emphasis on serving the privacy c
Introduction: You know your product is doing well when most of your early blog posts deal with the status of the waiting list of hundreds of thousands of users eagerly waiting to download your product. That's the enviable position Mailbox , a free mobile email management app, found themselves early in their release cycle. Hasn't email been done already? Apparently not. Mailbox scaled to one million users in a paltry six weeks with a team of about 14 people . As of April they were delivering over 100 million messages per day . How did they do it? Mailbox engineering lead, Sean Beausoleil , gave an informative interview on readwrite.com on how Mailbox planned to scale... Gather signals early . A pre-release launch video helped generate interest, but it also allowed them to gauge early interest before even releasing. From the overwhelming response they knew they would need to have to scale and scale quickly. Have something unique . The average person might not think a mailbox app
6 0.15735161 233 high scalability-2008-01-30-How Rackspace Now Uses MapReduce and Hadoop to Query Terabytes of Data
7 0.14693862 987 high scalability-2011-02-10-Dispelling the New SSL Myth
8 0.14349706 724 high scalability-2009-10-19-Drupal's Scalability Makeover - You give up some control and you get back scalability
9 0.12173943 30 high scalability-2007-07-26-Product: AWStats a Log Analyzer
10 0.11811613 253 high scalability-2008-02-19-Building a email communication system
11 0.10323445 202 high scalability-2008-01-06-Email Architecture
12 0.10277548 878 high scalability-2010-08-12-Strategy: Terminate SSL Connections in Hardware and Reduce Server Count by 40%
13 0.10245212 297 high scalability-2008-04-05-Skype Plans for PostgreSQL to Scale to 1 Billion Users
14 0.094182342 228 high scalability-2008-01-28-Product: ISPMan Centralized ISP Management System
15 0.090317018 29 high scalability-2007-07-25-Product: lighttpd
16 0.085829586 218 high scalability-2008-01-17-Moving old to new. Do not be afraid of the re-write -- but take some help
17 0.084488966 84 high scalability-2007-09-08-MP3.com Web Templating Architecture (March, 2000)
18 0.08384002 221 high scalability-2008-01-24-Mailinator Architecture
19 0.075669199 301 high scalability-2008-04-08-Google AppEngine - A First Look
20 0.073834613 1521 high scalability-2013-09-23-Salesforce Architecture - How they Handle 1.3 Billion Transactions a Day
topicId topicWeight
[(0, 0.097), (1, 0.036), (2, -0.026), (3, -0.044), (4, 0.01), (5, 0.009), (6, 0.015), (7, -0.054), (8, -0.009), (9, 0.024), (10, -0.003), (11, 0.05), (12, 0.034), (13, -0.016), (14, 0.039), (15, 0.005), (16, -0.018), (17, 0.035), (18, -0.005), (19, 0.017), (20, 0.048), (21, -0.018), (22, -0.096), (23, 0.006), (24, 0.031), (25, 0.041), (26, 0.015), (27, 0.002), (28, -0.031), (29, -0.025), (30, -0.033), (31, -0.024), (32, -0.064), (33, -0.069), (34, -0.013), (35, -0.037), (36, -0.041), (37, 0.047), (38, 0.051), (39, 0.018), (40, 0.046), (41, 0.07), (42, 0.053), (43, -0.031), (44, -0.049), (45, 0.032), (46, 0.022), (47, -0.063), (48, -0.002), (49, 0.014)]
simIndex simValue blogId blogTitle
same-blog 1 0.90662342 80 high scalability-2007-09-06-Product: Perdition Mail Retrieval Proxy
Introduction: Perdition is a fully featured POP3 and IMAP4 proxy server. It is able to handle both SSL and non-SSL connections and redirect users to a real-server based on a database lookup. Perdition supports modular based database access. ODBC, MySQL, PostgreSQL, GDBM, POSIX Regular Expression and NIS modules ship with the distribution. The API for modules is open allowing arbitrary modules to be written to allow access to any data store. Perdition has many uses. Including, creating large mail systems where an end-user's mailbox may be stored on one of several hosts, integrating different mail systems together, migrating between different email infrastructures, and bridging plain-text, SSL and TLS services. It can also be used as part of a firewall. The use of perditon to scale mail services beyond a single box is discussed in high capacity email.
2 0.73003381 57 high scalability-2007-08-03-Scaling IMAP and POP3
Introduction: Just thought I'd drop a brief suggestion to anyone building a large mail system. Our solution for scaling mail pickup was to develop a sharded architecture whereby accounts are spread across a cluster of servers, each with imap/pop3 capability. Then we use a cluster of reverse proxies (Perdition) speaking to the backend imap/pop3 servers . The benefit of this approach is you can use simply use round-robin or HA loadbalancing on the perdition servers that end users connect to (e.g. admins can easily move accounts around on the backend storage servers without affecting end users). Perdition manages routing users to the appropriate backend servers and has MySQL support. What we also liked about this approach was that it had no dependency on a distributed or networked filesystem, so less chance of corruption or data consistency issues. When an individual server reaches capacity, we just off load users to a less used server. If any server goes offline, it only affects the fraction of users
3 0.70875192 81 high scalability-2007-09-06-Scaling IMAP and POP3
Introduction: Another scalability strategy brought to you by Erik Osterman: Just thought I'd drop a brief suggestion to anyone building a large mail system. Our solution for scaling mail pickup was to develop a sharded architecture whereby accounts are spread across a cluster of servers, each with imap/pop3 capability. Then we use a cluster of reverse proxies (Perdition) speaking to the backend imap/pop3 servers . The benefit of this approach is you can use simply use round-robin or HA load balancing on the perdition servers that end users connect to (e.g. admins can easily move accounts around on the backend storage servers without affecting end users). Perdition manages routing users to the appropriate backend servers and has MySQL support. What we also liked about this approach was that it had no dependency on a distributed or networked file system, so less chance of corruption or data consistency issues. When an individual server reaches capacity, we just off load users to a less u
4 0.63291419 253 high scalability-2008-02-19-Building a email communication system
Introduction: hi, the website i work for is looking to build a email system that can handle a fair few emails (up to a hundred thousand a day). These comprise emails like registration emails, newsletters, lots of user triggered emails and overnight emails. At present we queue them in SQL and feed them into an smtp server on one of our web servers when the queue drops below a certain level. this has caused our mail system to crash as well as hammer our DB server (shared!!!). We have got an architecture of what we want to build but thought there might be something we could buy off the shelf that allowed us to keep templated emails, lists of recipients, schedule sends etc and report on it. We can't find anything What do big websites like amazon etc use or people a little smaller but who still send loads of mail (flickr, ebuyer, or other ecommerce sites) Cheers tarqs
5 0.62774223 1501 high scalability-2013-08-13-In Memoriam: Lavabit Architecture - Creating a Scalable Email Service
Introduction: With Lavabit shutting down under murky circumstances , it seems fitting to repost an old (2009), yet still very good post by Ladar Levison on Lavabit's architecture. I don't know how much of this information is still current, but it should give you a general idea what Lavabit was all about. Getting to Know You What is the name of your system and where can we find out more about it? Note: these links are no longer valid... Lavabit http://lavabit.com http://lavabit.com/network.html http://lavabit.com/about.html What is your system for? Lavabit is a mid-sized email service provider. We currently have about 140,000 registered users with more than 260,000 email addresses. While most of our accounts belong to individual users, we also provide corporate email services to approximately 70 companies. Why did you decide to build this system? We built the system to compete against the other large free email providers, with an emphasis on serving the privacy c
6 0.60293293 228 high scalability-2008-01-28-Product: ISPMan Centralized ISP Management System
7 0.58578765 551 high scalability-2009-03-30-Lavabit Architecture - Creating a Scalable Email Service
8 0.56908393 172 high scalability-2007-12-02-nginx: high performance smpt-pop-imap proxy
9 0.56506962 233 high scalability-2008-01-30-How Rackspace Now Uses MapReduce and Hadoop to Query Terabytes of Data
10 0.55954164 221 high scalability-2008-01-24-Mailinator Architecture
11 0.55587143 208 high scalability-2008-01-11-FTP Sanity: Redundancy, archiving, consolidation.
12 0.54353839 1196 high scalability-2012-02-20-Berkeley DB Architecture - NoSQL Before NoSQL was Cool
13 0.54067844 297 high scalability-2008-04-05-Skype Plans for PostgreSQL to Scale to 1 Billion Users
14 0.53767401 937 high scalability-2010-11-09-Paper: Hyder - Scaling Out without Partitioning
15 0.52873355 473 high scalability-2008-12-20-Second Life Architecture - The Grid
16 0.52594244 449 high scalability-2008-11-24-Product: Scribe - Facebook's Scalable Logging System
17 0.51821351 1390 high scalability-2013-01-21-Processing 100 Million Pixels a Day - Small Amounts of Contention Cause Big Problems at Scale
18 0.51564765 160 high scalability-2007-11-19-Tailrank Architecture - Learn How to Track Memes Across the Entire Blogosphere
19 0.5147438 1269 high scalability-2012-06-20-iDoneThis - Scaling an Email-based App from Scratch
20 0.50917584 1528 high scalability-2013-10-07-Ask HS: Is Microsoft the Right Technology for a Scalable Web-based System?
topicId topicWeight
[(1, 0.06), (2, 0.231), (38, 0.388), (61, 0.143), (94, 0.048)]
simIndex simValue blogId blogTitle
1 0.72906679 497 high scalability-2009-01-19-Papers: Readings in Distributed Systems
Introduction: Marton Trencseni has collected a wonderful list of different papers on distributed systems. He's organized them into the following sections: The Google Papers, Distributed Filesystems, Non-relational Distributed Databases, The Lamport Papers, and Implementation Issues. Many old favorites on the list and some that are likely new to you. My new favorite is "Frangipani: A Scalable Distributed File System." How can you not love "Frangipani" as a word?
same-blog 2 0.71095634 80 high scalability-2007-09-06-Product: Perdition Mail Retrieval Proxy
Introduction: Perdition is a fully featured POP3 and IMAP4 proxy server. It is able to handle both SSL and non-SSL connections and redirect users to a real-server based on a database lookup. Perdition supports modular based database access. ODBC, MySQL, PostgreSQL, GDBM, POSIX Regular Expression and NIS modules ship with the distribution. The API for modules is open allowing arbitrary modules to be written to allow access to any data store. Perdition has many uses. Including, creating large mail systems where an end-user's mailbox may be stored on one of several hosts, integrating different mail systems together, migrating between different email infrastructures, and bridging plain-text, SSL and TLS services. It can also be used as part of a firewall. The use of perditon to scale mail services beyond a single box is discussed in high capacity email.
3 0.67854583 1511 high scalability-2013-09-04-Wide Fast SATA: the Recipe for Hot Performance
Introduction: This is a guest post by Brian Bulkowski , CTO and co-founder of Aerospike , a leading clustered NoSQL database, has worked in the area of high performance commodity systems since 1989. This blog post will tell you exactly how to build a multi-terabyte high throughput datacenter server. A fast, reliable multi-terrabyte data tier can be used for recent behavior (messages, tweets, plays, actions), or anywhere that today you use Redis or Memcache. You need to know: Which SSDs work Which chassis work How to configure your RAID cards Intel’s SATA solutions – combined with a high capacity storage server like the Dell R720xd and a host bus adapter based on the LSI 2208, and a Flash optimized database like Aerospike , enables high throughput and low latency. In a wide configuration, with 12 to 20 drives per 2U server, individual servers can cost-effectively serve at high throughput with 16T at $2.50 per GB with the s3700, or $1.25 with the s3500. Other SSD of
4 0.63609874 772 high scalability-2010-02-05-High Availability Principle : Concurrency Control
Introduction: One important high availability principle is concurrency control. The idea is to allow only that much traffic through to your system which your system can handle successfully. For example: if your system is certified to handle a concurrency of 100 then the 101st request should either timeout, be asked to try later or wait until one of the previous 100 requests finish. The 101st request should not be allowed to negatively impact the experience of the other 100 users. Only the 101st request should be impacted. Read more here...
5 0.60606492 1452 high scalability-2013-05-06-7 Not So Sexy Tips for Saving Money On Amazon
Introduction: Harish Ganesan CTO of 8KMiles has a very helpful blog, Cloud, Big Data and Mobile , where he shows a nice analytical bent which leads to a lot of practical advice and cost saving tips: Use SQS Batch Requests to reduce the number of requests hitting SQS which saves costs. Sending 10 messages in a single batch request which in the example save $30/month. Use SQS Long Polling to reduce extra polling requests, cutting down empty receives, which in the example saves ~$600 in empty receive leakage costs. Choose the right search technology choice to save costs in AWS by matching your activity pattern to the technology. For a small application with constant load or a heavily utilized search tier or seasonal loads Amazon Cloud Search looks like the cost efficient play. Use Amazon CloudFront Price Class to minimize costs by selecting the right Price Class for your audience to potentially reduce delivery costs by excluding Amazon CloudFront’s more expensive edge locatio
6 0.60264766 1401 high scalability-2013-02-06-Super Bowl Advertisers Ready for the Traffic? Nope..It's Lights Out.
7 0.59811884 1121 high scalability-2011-09-21-5 Scalability Poisons and 3 Cloud Scalability Antidotes
8 0.57702893 994 high scalability-2011-02-23-This stuff isn't taught, you learn it bit by bit as you solve each problem.
9 0.5764035 1491 high scalability-2013-07-15-Ask HS: What's Wrong with Twitter, Why Isn't One Machine Enough?
10 0.57224709 1473 high scalability-2013-06-10-The 10 Deadly Sins Against Scalability
11 0.56716573 89 high scalability-2007-09-10-Is there a difference between partitioning and federation and sharding?
12 0.56673098 685 high scalability-2009-08-20-Dependency Injection and AOP frameworks for .NET
13 0.56652725 1135 high scalability-2011-10-31-15 Ways to Make Your Application Feel More Responsive under Google App Engine
14 0.56596738 877 high scalability-2010-08-12-Designing Web Applications for Scalability
15 0.56279922 564 high scalability-2009-04-10-counting # of views, calculating most-least viewed
16 0.55632842 1538 high scalability-2013-10-28-Design Decisions for Scaling Your High Traffic Feeds
17 0.55564398 943 high scalability-2010-11-16-Facebook's New Real-time Messaging System: HBase to Store 135+ Billion Messages a Month
18 0.55350405 1172 high scalability-2012-01-10-A Perfect Fifth of Notes on Scalability
19 0.55282766 151 high scalability-2007-11-12-a8cjdbc - Database Clustering via JDBC
20 0.54991466 437 high scalability-2008-11-03-How Sites are Scaling Up for the Election Night Crush