high_scalability high_scalability-2008 high_scalability-2008-435 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Hi all, I'm a big fan of http://highscalability.com/ and have been looking in my current development to decompose my application along functional boundaries as a route to being able to scale out the server side, specifically the database layer. The problem comes when there are links between the data in different components, ie one component holds all the user data, but another component needs to reference a user as being an owner of some piece of data. I'm currently doing this by holding the primary key information for each side of the the link (as you would if they all lived in a single database), but this link table needs to exist in both components to allow lookups to be done in either direction, ie 'get the things a specific user owns' and 'get the owners of this specific thing' would each use different components. The alternative to this would be to store the link data in only one of the components, but then the reverse lookups would require 2 calls instead of just one. M
sentIndex sentText sentNum sentScore
1 Hi all, I'm a big fan of http://highscalability. [sent-1, score-0.126]
2 com/ and have been looking in my current development to decompose my application along functional boundaries as a route to being able to scale out the server side, specifically the database layer. [sent-2, score-0.827]
3 The problem comes when there are links between the data in different components, ie one component holds all the user data, but another component needs to reference a user as being an owner of some piece of data. [sent-3, score-1.481]
4 The alternative to this would be to store the link data in only one of the components, but then the reverse lookups would require 2 calls instead of just one. [sent-5, score-1.083]
5 My question is this, is the duplication of these link tables some kind of code smell I should be avoiding or is this just the way things go when you split your app along functional lines like this? [sent-6, score-1.526]
6 Is this sort of approach really applicable to anyone other than the ebays of this world? [sent-7, score-0.284]
7 should the rest of us just keep putting more functionality into the same back end? [sent-8, score-0.234]
wordName wordTfidf (topN-words)
[('link', 0.351), ('ie', 0.302), ('lookups', 0.242), ('components', 0.193), ('functional', 0.182), ('component', 0.167), ('smell', 0.166), ('duplication', 0.162), ('decompose', 0.162), ('along', 0.15), ('owns', 0.149), ('lived', 0.149), ('hi', 0.143), ('owners', 0.141), ('specific', 0.138), ('applicable', 0.137), ('holding', 0.137), ('side', 0.131), ('holds', 0.131), ('boundaries', 0.131), ('fan', 0.126), ('user', 0.116), ('direction', 0.113), ('would', 0.108), ('avoiding', 0.107), ('route', 0.105), ('needs', 0.105), ('reverse', 0.105), ('reference', 0.103), ('exist', 0.098), ('piece', 0.097), ('specifically', 0.097), ('split', 0.09), ('alternative', 0.09), ('owner', 0.09), ('putting', 0.087), ('links', 0.087), ('lines', 0.087), ('things', 0.081), ('tables', 0.08), ('calls', 0.079), ('primary', 0.078), ('functionality', 0.077), ('either', 0.075), ('anyone', 0.075), ('sort', 0.072), ('table', 0.072), ('rest', 0.07), ('question', 0.07), ('currently', 0.066)]
simIndex simValue blogId blogTitle
same-blog 1 1.0000001 435 high scalability-2008-10-30-The case for functional decomposition
Introduction: Hi all, I'm a big fan of http://highscalability.com/ and have been looking in my current development to decompose my application along functional boundaries as a route to being able to scale out the server side, specifically the database layer. The problem comes when there are links between the data in different components, ie one component holds all the user data, but another component needs to reference a user as being an owner of some piece of data. I'm currently doing this by holding the primary key information for each side of the the link (as you would if they all lived in a single database), but this link table needs to exist in both components to allow lookups to be done in either direction, ie 'get the things a specific user owns' and 'get the owners of this specific thing' would each use different components. The alternative to this would be to store the link data in only one of the components, but then the reverse lookups would require 2 calls instead of just one. M
2 0.15146087 119 high scalability-2007-10-10-WAN Accelerate Your Way to Lightening Fast Transfers Between Data Centers
Introduction: How do you keep in sync a crescendo of data between data centers over a slow WAN? That's the question Alberto posted a few weeks ago. Normally I'm not into all boy bands, but I was frustrated there wasn't a really good answer for his problem. It occurred to me later a WAN accelerator might help turn his slow WAN link into more of a LAN, so the overhead of copying files across the WAN wouldn't be so limiting. Many might not consider a WAN accelerator in this situation, but since my friend Damon Ennis works at the WAN accelerator vendor Silver Peak , I thought I would ask him if their product would help. Not surprisingly his answer is yes! Potentially a lot, depending on the nature of your data. Here's a no BS overview of their product: What is it? - Scalable WAN Accelerator from Silver Peak (http://www.silver-peak.com) What does it do? - You can send 5x-100x times more data across your expensive, low-bandwidth WAN link. Why should you care? - Your data centers becom
3 0.13091058 653 high scalability-2009-07-08-Servers Component - How to choice and build perfect server
Introduction: There are a lot of questions about how the server components, and how to build perfect server with consider the power consumption. Today I will discuss the Server components, and how we can choice better server components with consider the power consumption, efficacy, performance, and price. Key points: What kind of components the servers needs? The Green Computing and the Servers components How much power the server consume Choice the right components: Processor Hard Disk Drive Memory Operating system Build Server, or buy?
4 0.12751062 828 high scalability-2010-05-17-7 Lessons Learned While Building Reddit to 270 Million Page Views a Month
Introduction: Steve Huffman , co-founder of social news site Reddit , gave an excellent presentation ( slides , transcript ) on the lessons he learned while building and growing Reddit to 7.5 million users per month, 270 million page views per month, and 20+ database servers. Steve says a lot of the lessons were really obvious, so you may not find a lot of completely new ideas in the presentation. But Steve has an earnestness and genuineness about him that is so obviously grounded in experience that you can't help but think deeply about what you could be doing different. And if Steve didn't know about these lessons, I'm betting others don't either. There are seven lessons, each has their own summary section: Lesson one: Crash Often; Lesson 2: Separation of Services; Lesson 3: Open Schema; Lesson 4: Keep it Stateless; Lesson 5: Memcache; Lesson 6: Store Redundant Data; Lesson 7: Work Offline. By far the most surprising feature of their architecture is in Lesson Six, whose essential idea is:
5 0.11194026 906 high scalability-2010-09-22-Applying Scalability Patterns to Infrastructure Architecture
Introduction: Too often software design patterns are overlooked by network and application delivery network architects but these patterns are often equally applicable to addressing a broad range of architectural challenges in the application delivery tier of the data center. By Lori Mac Vittie, F5 Networks The “ High Scalability ” blog is fast becoming one of my favorite reads. Last week did not disappoint with a post highlighting a set of scalability design patterns that was, apparently, inspired by yet another High Scalability post on “ 6 Ways to Kill Your Servers: Learning to Scale the Hard Way. ” Credit:Michael Chow/azcentral.com This particular post caught my attention primarily because although I’ve touched on many of these patterns in the past, I’ve never thought to call them what they are: scalability patterns. That’s probably a side-effect of forgetting that building an architecture of any kind is at its core computer science and thus
6 0.1070896 54 high scalability-2007-08-02-Multilanguage Website
7 0.10674014 1440 high scalability-2013-04-15-Scaling Pinterest - From 0 to 10s of Billions of Page Views a Month in Two Years
8 0.1057594 585 high scalability-2009-04-29-How to choice and build perfect server
9 0.10391966 741 high scalability-2009-11-16-Building Scalable Systems Using Data as a Composite Material
10 0.09968102 1501 high scalability-2013-08-13-In Memoriam: Lavabit Architecture - Creating a Scalable Email Service
11 0.099363737 428 high scalability-2008-10-24-11 Secrets of a Cloud Scale Consultant That They Dont' Want You to Know
12 0.097489856 1421 high scalability-2013-03-11-Low Level Scalability Solutions - The Conditioning Collection
13 0.09735778 897 high scalability-2010-09-08-4 General Core Scalability Patterns
14 0.093424797 840 high scalability-2010-06-10-The Four Meta Secrets of Scaling at Facebook
15 0.08881586 910 high scalability-2010-09-30-Facebook and Site Failures Caused by Complex, Weakly Interacting, Layered Systems
18 0.085116535 1565 high scalability-2013-12-16-22 Recommendations for Building Effective High Traffic Web Software
19 0.083056554 620 high scalability-2009-06-05-SSL RPC API Scalability
20 0.082575195 532 high scalability-2009-03-11-Sharding and Connection Pools
topicId topicWeight
[(0, 0.147), (1, 0.061), (2, -0.035), (3, -0.041), (4, 0.03), (5, -0.007), (6, 0.012), (7, -0.02), (8, -0.006), (9, -0.001), (10, -0.017), (11, 0.065), (12, -0.039), (13, -0.006), (14, 0.068), (15, -0.031), (16, -0.01), (17, 0.024), (18, 0.016), (19, -0.002), (20, -0.017), (21, -0.051), (22, -0.039), (23, 0.002), (24, 0.029), (25, -0.035), (26, 0.038), (27, -0.053), (28, 0.013), (29, 0.033), (30, -0.042), (31, 0.021), (32, -0.058), (33, 0.016), (34, 0.027), (35, 0.034), (36, 0.095), (37, -0.014), (38, -0.023), (39, -0.01), (40, 0.034), (41, 0.028), (42, -0.068), (43, -0.026), (44, 0.036), (45, -0.01), (46, 0.001), (47, 0.039), (48, -0.053), (49, 0.081)]
simIndex simValue blogId blogTitle
same-blog 1 0.9545958 435 high scalability-2008-10-30-The case for functional decomposition
Introduction: Hi all, I'm a big fan of http://highscalability.com/ and have been looking in my current development to decompose my application along functional boundaries as a route to being able to scale out the server side, specifically the database layer. The problem comes when there are links between the data in different components, ie one component holds all the user data, but another component needs to reference a user as being an owner of some piece of data. I'm currently doing this by holding the primary key information for each side of the the link (as you would if they all lived in a single database), but this link table needs to exist in both components to allow lookups to be done in either direction, ie 'get the things a specific user owns' and 'get the owners of this specific thing' would each use different components. The alternative to this would be to store the link data in only one of the components, but then the reverse lookups would require 2 calls instead of just one. M
2 0.71272266 532 high scalability-2009-03-11-Sharding and Connection Pools
Introduction: Hi we are looking at sharding our existing Java/Oracle based application. We are looking to make the app servers able to process requests for multiple (any?) shard. The concern that has come up is the amount of memory that would be consumed by having so many connection pools on one app server. Additionally there is concern about having so many physical connections to the database server coming from all the various app servers that may talk to that particular shard. I was wondering if anyone else has dealt with this issue and how you resolved it? Thanks, Scott
3 0.71075565 379 high scalability-2008-09-04-Database question for upcoming project
Introduction: We will be developing an RIA that will have a lot of database access. Think something like a QuickBooks but with about 50 transactions entered per hour per user. Users will be in the system for 7 to 9 hours a day and there will be around 20,000 users, all logged in at the same time. Reporting will be done just like a QuickBooks style app plus a lot of extra things you don't do in QuickBooks. Our operations is familiar with W2003 Server and MS SQL Server so they are recommending we stick with that. I originally requested Linux and PostgreSQL. How far can a single database server get me? If we have a 4 processor, 8 core, 128gb server, how far am I going to get before I need to shard or do something else? I know there are a lot of factors involved but in general for this size of a site, what should the strategy be? I've read almost all articles on this website but most of the applications are not RIA type of apps with this type of usage or they are architectures for
4 0.70692551 675 high scalability-2009-08-08-1dbase vs. many and cloud hosting vs. dedicated server(s)?
Introduction: Me and my partner are making a blueprint for an online webshop service. The purpose of this project is to make webshops available for small company's/ individuals automatically just by creating an account with us. Our webapp can be used to add products/pages/... to the store and we'll handle secure checkout by paypal. Our app should be scalable and manageable. Because we also want to offer free webshops, the amount of webshops could be +10.000 within a few years. We are building on the Zend framework and are using mysql for database. From the start we want to build our application for optimal and easy scalability in the future, to avoid a lot changes to our app/database in the future. Now our questions are: Should we use?: * one database for all shops (or limited to X shops ); * one database for each new shop (each having products, orders... tables); I think both approaches have PRO/CONS. What do you think ? Does anyone has experience with this kind of structure ? PRO: one datab
5 0.7037195 1579 high scalability-2014-01-14-SharePoint VPS solution
Introduction: Launched in 2001 by Microsoft, since then this platform is widely popular among organizations to increase the efficiency of their business and productive volume. As we all know the importance of communication mode in every organization and by keeping in the mind this fact, Microsoft SharePoint communicates over all barriers and spread the required information within a company. Microsoft SharePoint is an ideal solution for companies who have multiple offices and staff members who are on the move. Using SharePoint, documents and other materials can be easily shared with both colleagues and managers. Other features include advanced document management, which allows users to virtually check out a document, modify it or just read it, then check in the document again. This allows managers/company owners to see exactly when their staff members are working and just what they are doing. When combined with a highly customizable workflow management system and group calendars, SharePoint can imp
6 0.69778728 451 high scalability-2008-11-30-Creating a high-performing online database
7 0.69204175 369 high scalability-2008-08-18-Code deployment tools
8 0.68358338 1268 high scalability-2012-06-20-Ask HighScalability: How do I organize millions of images?
9 0.67891747 222 high scalability-2008-01-25-Application Database and DAL Architecture
10 0.66325194 1650 high scalability-2014-05-19-A Short On How the Wayback Machine Stores More Pages than Stars in the Milky Way
11 0.66111428 606 high scalability-2009-05-25-non-sequential, unique identifier, strategy question
12 0.6547851 683 high scalability-2009-08-18-Hardware Architecture Example (geographical level mapping of servers)
13 0.65153629 177 high scalability-2007-12-08-thesimsonstage.ea.com
14 0.65074551 828 high scalability-2010-05-17-7 Lessons Learned While Building Reddit to 270 Million Page Views a Month
15 0.64144373 561 high scalability-2009-04-08-N+1+caching is ok?
16 0.63922238 256 high scalability-2008-02-21-Tracking usage of public resources - throttling accesses per hour
17 0.63443911 578 high scalability-2009-04-23-Which Key value pair database to be used
18 0.63372171 1288 high scalability-2012-07-23-Ask HighScalability: How Do I Build My MegaUpload + Itunes + YouTube Startup?
19 0.63048047 993 high scalability-2011-02-22-Is Node.js Becoming a Part of the Stack? SimpleGeo Says Yes.
20 0.62785476 266 high scalability-2008-03-04-Manage Downtime Risk by Connecting Multiple Data Centers into a Secure Virtual LAN
topicId topicWeight
[(1, 0.175), (2, 0.168), (61, 0.168), (79, 0.083), (94, 0.07), (96, 0.231)]
simIndex simValue blogId blogTitle
same-blog 1 0.88951975 435 high scalability-2008-10-30-The case for functional decomposition
Introduction: Hi all, I'm a big fan of http://highscalability.com/ and have been looking in my current development to decompose my application along functional boundaries as a route to being able to scale out the server side, specifically the database layer. The problem comes when there are links between the data in different components, ie one component holds all the user data, but another component needs to reference a user as being an owner of some piece of data. I'm currently doing this by holding the primary key information for each side of the the link (as you would if they all lived in a single database), but this link table needs to exist in both components to allow lookups to be done in either direction, ie 'get the things a specific user owns' and 'get the owners of this specific thing' would each use different components. The alternative to this would be to store the link data in only one of the components, but then the reverse lookups would require 2 calls instead of just one. M
2 0.88696623 281 high scalability-2008-03-18-Database Design 101
Introduction: I am working on the design for my database and can't seem to come up with a firm schema. I am torn between normalizing the data and dealing with the overhead of joins and denormalizing it for easy sharding. The data is essentially music information per user: UserID, Artist, Album, Song. This lends itself nicely to be normalized and have separate User, Artist, Album and Song databases with a table full of INTs to tie them together. This will be in a mostly read based environment and with about 80% being searches of data by artist album or song. By the time I begin the query for artist, album or song I will already have a list of UserID's to limit the search by. The problem is that the tables can get unmanageably large pretty quickly and my plan was to shard off users once it got too big. Given this simple data relationship what are the pros and cons of normalizing the data vs denormalizing it? Should I go with 4 separate, normalized tables or one 4 column table? Perhaps it might
3 0.88027799 1528 high scalability-2013-10-07-Ask HS: Is Microsoft the Right Technology for a Scalable Web-based System?
Introduction: This question was asked over email and I thought a larger audience might want to take a whack at it. I have a problem I’d like to have your view on. I’ve looked around a lot, and I haven’t found a definite answer. The question is this: Is it true that for a scalable web-based system targeting millions of users (hopefully), using Microsoft technology(.Net/SQL Server) over open source technologies like python/ruby/php and mysql (mariadb) / postgresql will cost you more? Is there any justification for paying up for Microsoft licenses(OS, SQL Server, …)? I am in charge of selecting the technology toolbox for a startup which is going to build a scalable public web platform. I’ve worked as a developer and database developer/admin (mainly as a DBA) using different platforms and technologies, but my main focus is on Microsoft technology. I’ve considered all other important factors for this decision, and at the end, I always come back to the question of money. When I finish developing th
4 0.86293209 348 high scalability-2008-07-09-Federation at Flickr: Doing Billions of Queries Per Day
Introduction: Flickr's lone database guy Dathan Pattishall made his excellent presentation available on how on how Flickr scales its backend to handle tremendous loads. Some of this information is available in Flickr Architecture , but the paper is so good it's worth another read. If you want to see sharding done right, at scale, take a look.
5 0.84510982 1212 high scalability-2012-03-21-The Conspecific Hybrid Cloud
Introduction: When you’re looking to add new tank mates to an existing aquarium ecosystem, one of the concerns you must have is whether a particular breed of fish is amenable to conspecific cohabitants. Many species are not, which means if you put them together in a confined space, they’re going to fight. Viciously. To the death. Responsible aquarists try to avoid such situations, so careful attention to the conspecificity of animals is a must. Now, while in many respects the data center ecosystem correlates well to an aquarium ecosystem, in this case it does not. It’s what you usually get, today, but its not actually the best model. That’s because what you want in the data center ecosystem – particularly when it extends to include public cloud computing resources – is conspecificity in infrastructure. This desire and practice is being seen both in enterprise data center decision making as well as in startups suddenly dealing with massive growth and increasingly encountering pe
6 0.84453928 1549 high scalability-2013-11-15-Stuff The Internet Says On Scalability For November 15th, 2013
7 0.8379578 162 high scalability-2007-11-20-what is j2ee stack
8 0.82840919 1052 high scalability-2011-06-03-Stuff The Internet Says On Scalability For June 3, 2011
9 0.81459731 422 high scalability-2008-10-17-Scaling Spam Eradication Using Purposeful Games: Die Spammer Die!
10 0.8110708 1228 high scalability-2012-04-16-Instagram Architecture Update: What’s new with Instagram?
11 0.80174279 397 high scalability-2008-09-28-Product: Happy = Hadoop + Python
12 0.80021471 828 high scalability-2010-05-17-7 Lessons Learned While Building Reddit to 270 Million Page Views a Month
13 0.79854202 868 high scalability-2010-07-30-Basho Lives up to their Name With Consistent Smashing
14 0.78950536 1035 high scalability-2011-05-05-Paper: A Study of Practical Deduplication
15 0.78801894 775 high scalability-2010-02-10-ElasticSearch - Open Source, Distributed, RESTful Search Engine
16 0.78419966 931 high scalability-2010-10-28-Notes from A NOSQL Evening in Palo Alto
17 0.78398627 598 high scalability-2009-05-12-P2P server technology?
18 0.78199965 810 high scalability-2010-04-14-Parallel Information Retrieval and Other Search Engine Goodness
19 0.78053045 1289 high scalability-2012-07-23-State of the CDN: More Traffic, Stable Prices, More Products, Profits - Not So Much
20 0.77872092 787 high scalability-2010-03-03-Hot Scalability Links for March 3, 2010