high_scalability high_scalability-2009 high_scalability-2009-578 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: My Table has 2 columsn .Column1 is id,Column2 contains information given by user about item in Column1 .User can give 3 types of information about item.I separate the opinion of single user by comma,and opinion of another user by ;. Example- 23-34,us,56;78,in,78 I need to calculate opinions of all users very fast.My idea is to have index on key so the searching would be very fast.Currently i m using mysql .My problem is that maximum column size is below my requirement .If any overflow occurs i make new row with same id and insert data into new row. Practically I would have around maximum 5-10 for each row. I think if there is any database which removes this application code. I just learn about key value pair database which is exactly i needed . But which doesn't put constraint(i mean much better than RDMS on column size. This application is not in production.
sentIndex sentText sentNum sentScore
1 Column1 is id,Column2 contains information given by user about item in Column1 . [sent-2, score-0.665]
2 I separate the opinion of single user by comma,and opinion of another user by ;. [sent-4, score-1.294]
3 Example- 23-34,us,56;78,in,78 I need to calculate opinions of all users very fast. [sent-5, score-0.427]
4 My idea is to have index on key so the searching would be very fast. [sent-6, score-0.515]
5 My problem is that maximum column size is below my requirement . [sent-8, score-0.88]
6 If any overflow occurs i make new row with same id and insert data into new row. [sent-9, score-0.907]
7 Practically I would have around maximum 5-10 for each row. [sent-10, score-0.406]
8 I think if there is any database which removes this application code. [sent-11, score-0.369]
9 I just learn about key value pair database which is exactly i needed . [sent-12, score-0.708]
10 But which doesn't put constraint(i mean much better than RDMS on column size. [sent-13, score-0.583]
wordName wordTfidf (topN-words)
[('opinion', 0.386), ('column', 0.32), ('rdms', 0.273), ('maximum', 0.268), ('constraint', 0.21), ('opinions', 0.206), ('overflow', 0.196), ('pair', 0.175), ('calculate', 0.172), ('removes', 0.167), ('item', 0.163), ('user', 0.161), ('insert', 0.156), ('requirement', 0.156), ('occurs', 0.153), ('row', 0.145), ('searching', 0.145), ('contains', 0.138), ('key', 0.119), ('information', 0.119), ('index', 0.11), ('id', 0.107), ('exactly', 0.105), ('table', 0.1), ('types', 0.1), ('mean', 0.097), ('value', 0.09), ('separate', 0.089), ('size', 0.085), ('given', 0.084), ('needed', 0.08), ('application', 0.08), ('production', 0.076), ('would', 0.075), ('database', 0.071), ('put', 0.071), ('give', 0.069), ('learn', 0.068), ('idea', 0.066), ('mysql', 0.064), ('another', 0.063), ('around', 0.063), ('new', 0.057), ('problem', 0.051), ('better', 0.051), ('think', 0.051), ('users', 0.049), ('single', 0.048), ('much', 0.044), ('make', 0.036)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999988 578 high scalability-2009-04-23-Which Key value pair database to be used
Introduction: My Table has 2 columsn .Column1 is id,Column2 contains information given by user about item in Column1 .User can give 3 types of information about item.I separate the opinion of single user by comma,and opinion of another user by ;. Example- 23-34,us,56;78,in,78 I need to calculate opinions of all users very fast.My idea is to have index on key so the searching would be very fast.Currently i m using mysql .My problem is that maximum column size is below my requirement .If any overflow occurs i make new row with same id and insert data into new row. Practically I would have around maximum 5-10 for each row. I think if there is any database which removes this application code. I just learn about key value pair database which is exactly i needed . But which doesn't put constraint(i mean much better than RDMS on column size. This application is not in production.
2 0.16733629 589 high scalability-2009-05-05-Drop ACID and Think About Data
Introduction: The abstract for the talk given by Bob Ippolito, co-founder and CTO of Mochi Media, Inc: Building large systems on top of a traditional single-master RDBMS data storage layer is no longer good enough. This talk explores the landscape of new technologies available today to augment your data layer to improve performance and reliability. Is your application a good fit for caches, bloom filters, bitmap indexes, column stores, distributed key/value stores, or document databases? Learn how they work (in theory and practice) and decide for yourself. Bob does an excellent job highlighting different products and the key concepts to understand when pondering the wide variety of new database offerings. It's unlikely you'll be able to say oh, this is the database for me after watching the presentation, but you will be much better informed on your options. And I imagine slightly confused as to what to do :-) An interesting observation in the talk is that the more robust products are internal
3 0.16372018 342 high scalability-2008-06-08-Search fast in million rows
Introduction: I have a table .This table has many columns but search performed based on 1 columns ,this table can have more than million rows. The data in these columns is something like funny,new york,hollywood User can search with parameters as funny hollywood .I need to take this 2 words and then search on column whether that column contain this words and how many times .It is not possible to index here .If the results return say 1200 results then without comparing each and every column i can't determine no of results.I need to compare for each and every column.This query is very frequent .How can i approach for this problem.What type of architecture,tools is helpful. I just know that this can be accomplished with distributed system but how can i make this system. I also see in this website that LinkedIn uses Lucene for search .Is Lucene is helpful in my case.My table has also lots of insertion ,however updation in not very frequent.
4 0.14911509 998 high scalability-2011-03-03-Stack Overflow Architecture Update - Now at 95 Million Page Views a Month
Introduction: A lot has happened since my first article on the Stack Overflow Architecture . Contrary to the theme of that last article, which lavished attention on Stack Overflow's dedication to a scale-up strategy, Stack Overflow has both grown up and out in the last few years. Stack Overflow has grown up by more then doubling in size to over 16 million users and multiplying its number of page views nearly 6 times to 95 million page views a month. Stack Overflow has grown out by expanding into the Stack Exchange Network , which includes Stack Overflow, Server Fault, and Super User for a grand total of 43 different sites. That's a lot of fruitful multiplying going on. What hasn't changed is Stack Overflow's openness about what they are doing. And that's what prompted this update. A recent series of posts talks a lot about how they've been handling their growth: Stack Exchange’s Architecture in Bullet Points , Stack Overflow’s New York Data Center , Designing For Scalability of Manageme
5 0.13402219 671 high scalability-2009-08-05-Stack Overflow Architecture
Introduction: Update 2 : Stack Overflow Architecture Update - Now At 95 Million Page Views A Month Update: Startup – ASP.NET MVC, Cloud Scale & Deployment shows an interesting alternative approach for a Windows stack using ServerPath/GoGrid for a dedicated database machine, elastic VMs for the front end, and a free load balancer. Stack Overflow is a much loved programmer question and answer site written by two guys nobody has ever heard of before. Well, not exactly. The site was created by top programmer and blog stars Jeff Atwood and Joel Spolsky . In that sense Stack Overflow is like a celebrity owned restaurant, only it should be around for a while. Joel estimates 1/3 of all the programmers in the world have used the site so they must be serving up something good. I fell in deep like with Stack Overflow for purely selfish reasons, it helped me solve a few difficult problems that were jabbing my eyes out with pain. I also appreciate their no-apologies anthropologically based desig
6 0.1154722 276 high scalability-2008-03-15-New Website Design Considerations
7 0.11206662 11 high scalability-2007-07-15-Coyote Point Load Balancing Systems
9 0.11066354 658 high scalability-2009-07-17-Against all the odds
10 0.10478796 110 high scalability-2007-10-03-Why most large-scale Web sites are not written in Java
11 0.10213662 472 high scalability-2008-12-19-How to measure memory required for a user session
12 0.097529635 476 high scalability-2008-12-28-How to Organize a Database Table’s Keys for Scalability
13 0.093831137 1065 high scalability-2011-06-21-Running TPC-C on MySQL-RDS
14 0.090277679 954 high scalability-2010-12-06-What the heck are you actually using NoSQL for?
15 0.089639828 829 high scalability-2010-05-20-Strategy: Scale Writes to 734 Million Records Per Day Using Time Partitioning
16 0.085483164 1093 high scalability-2011-08-05-Stuff The Internet Says On Scalability For August 5, 2011
17 0.084039316 1131 high scalability-2011-10-24-StackExchange Architecture Updates - Running Smoothly, Amazon 4x More Expensive
18 0.083226286 673 high scalability-2009-08-07-Strategy: Break Up the Memcache Dog Pile
19 0.081698142 1080 high scalability-2011-07-15-Stuff The Internet Says On Scalability For July 15, 2011
topicId topicWeight
[(0, 0.122), (1, 0.047), (2, -0.021), (3, -0.062), (4, 0.019), (5, 0.066), (6, -0.017), (7, -0.02), (8, 0.041), (9, -0.054), (10, -0.001), (11, 0.045), (12, -0.041), (13, 0.025), (14, 0.039), (15, 0.006), (16, -0.061), (17, -0.031), (18, 0.005), (19, -0.008), (20, -0.008), (21, -0.043), (22, -0.012), (23, 0.033), (24, -0.006), (25, -0.012), (26, -0.019), (27, -0.002), (28, 0.043), (29, 0.04), (30, -0.022), (31, 0.024), (32, 0.005), (33, 0.02), (34, 0.011), (35, 0.015), (36, 0.011), (37, -0.019), (38, 0.014), (39, -0.057), (40, 0.033), (41, -0.009), (42, -0.015), (43, 0.038), (44, -0.037), (45, 0.035), (46, -0.005), (47, 0.026), (48, -0.007), (49, 0.039)]
simIndex simValue blogId blogTitle
same-blog 1 0.96220541 578 high scalability-2009-04-23-Which Key value pair database to be used
Introduction: My Table has 2 columsn .Column1 is id,Column2 contains information given by user about item in Column1 .User can give 3 types of information about item.I separate the opinion of single user by comma,and opinion of another user by ;. Example- 23-34,us,56;78,in,78 I need to calculate opinions of all users very fast.My idea is to have index on key so the searching would be very fast.Currently i m using mysql .My problem is that maximum column size is below my requirement .If any overflow occurs i make new row with same id and insert data into new row. Practically I would have around maximum 5-10 for each row. I think if there is any database which removes this application code. I just learn about key value pair database which is exactly i needed . But which doesn't put constraint(i mean much better than RDMS on column size. This application is not in production.
2 0.78341055 281 high scalability-2008-03-18-Database Design 101
Introduction: I am working on the design for my database and can't seem to come up with a firm schema. I am torn between normalizing the data and dealing with the overhead of joins and denormalizing it for easy sharding. The data is essentially music information per user: UserID, Artist, Album, Song. This lends itself nicely to be normalized and have separate User, Artist, Album and Song databases with a table full of INTs to tie them together. This will be in a mostly read based environment and with about 80% being searches of data by artist album or song. By the time I begin the query for artist, album or song I will already have a list of UserID's to limit the search by. The problem is that the tables can get unmanageably large pretty quickly and my plan was to shard off users once it got too big. Given this simple data relationship what are the pros and cons of normalizing the data vs denormalizing it? Should I go with 4 separate, normalized tables or one 4 column table? Perhaps it might
3 0.76636916 64 high scalability-2007-08-10-How do we make a large real-time search engine?
Introduction: We're implementing a website which should be oriented to content and with massive access by public and we would need a search engine to index and execute queries on the indexes of contents (stored in a database, most likely MySQL InnoDB or Oracle). The solution we found is to implement a separate service to make index constantly the contents of the database at regular intervals. Anyway, this is a complex and not optimal solution, since we would like it to index in real time and make it searchable. Could you point me to some examples or articles I could review to design a solution for such this context?
4 0.76458836 1233 high scalability-2012-04-25-The Anatomy of Search Technology: blekko’s NoSQL database
Introduction: This is a guest post ( part 2 , part 3 ) by Greg Lindahl, CTO of blekko, the spam free search engine that had over 3.5 million unique visitors in March. Greg Lindahl was Founder and Distinguished Engineer at PathScale, at which he was the architect of the InfiniPath low-latency InfiniBand HCA, used to build tightly-coupled supercomputing clusters. Imagine that you're crazy enough to think about building a search engine. It's a huge task: the minimum index size needed to answer most queries is a few billion webpages. Crawling and indexing a few billion webpages requires a cluster with several petabytes of usable disk -- that's several thousand 1 terabyte disks -- and produces an index that's about 100 terabytes in size. Serving query results quickly involves having most of the index in RAM or on solid state (flash) disk. If you can buy a server with 100 gigabytes of RAM for about $3,000, that's 1,000 servers at a capital cost of $3 million, plus about $1 million per year of serve
5 0.75449872 246 high scalability-2008-02-12-Search the tags across all post
Introduction: Let suppose i have table which stored tags .Now user can enter keywords and i have to search through all the records in table and find post which contain tags entered by user .user can enter more than 1 keywords. What strategy ,technique i use to search fast .There maybe more than millions records and many users are firing same query. Thanks
6 0.75355464 1065 high scalability-2011-06-21-Running TPC-C on MySQL-RDS
7 0.72899997 1440 high scalability-2013-04-15-Scaling Pinterest - From 0 to 10s of Billions of Page Views a Month in Two Years
8 0.72403836 342 high scalability-2008-06-08-Search fast in million rows
9 0.72182643 828 high scalability-2010-05-17-7 Lessons Learned While Building Reddit to 270 Million Page Views a Month
10 0.71641421 986 high scalability-2011-02-10-Database Isolation Levels And Their Effects on Performance and Scalability
11 0.70612389 587 high scalability-2009-05-01-FastBit: An Efficient Compressed Bitmap Index Technology
12 0.69805044 451 high scalability-2008-11-30-Creating a high-performing online database
13 0.69628751 351 high scalability-2008-07-16-The Mother of All Database Normalization Debates on Coding Horror
14 0.69560754 675 high scalability-2009-08-08-1dbase vs. many and cloud hosting vs. dedicated server(s)?
15 0.69163382 1650 high scalability-2014-05-19-A Short On How the Wayback Machine Stores More Pages than Stars in the Milky Way
16 0.67485303 1304 high scalability-2012-08-14-MemSQL Architecture - The Fast (MVCC, InMem, LockFree, CodeGen) and Familiar (SQL)
17 0.66647995 151 high scalability-2007-11-12-a8cjdbc - Database Clustering via JDBC
18 0.66487443 222 high scalability-2008-01-25-Application Database and DAL Architecture
19 0.66403443 435 high scalability-2008-10-30-The case for functional decomposition
20 0.66212642 65 high scalability-2007-08-16-Scaling Secret #2: Denormalizing Your Way to Speed and Profit
topicId topicWeight
[(1, 0.214), (2, 0.174), (10, 0.062), (44, 0.264), (47, 0.031), (94, 0.124)]
simIndex simValue blogId blogTitle
same-blog 1 0.79161286 578 high scalability-2009-04-23-Which Key value pair database to be used
Introduction: My Table has 2 columsn .Column1 is id,Column2 contains information given by user about item in Column1 .User can give 3 types of information about item.I separate the opinion of single user by comma,and opinion of another user by ;. Example- 23-34,us,56;78,in,78 I need to calculate opinions of all users very fast.My idea is to have index on key so the searching would be very fast.Currently i m using mysql .My problem is that maximum column size is below my requirement .If any overflow occurs i make new row with same id and insert data into new row. Practically I would have around maximum 5-10 for each row. I think if there is any database which removes this application code. I just learn about key value pair database which is exactly i needed . But which doesn't put constraint(i mean much better than RDMS on column size. This application is not in production.
2 0.71842855 241 high scalability-2008-02-05-SLA monitoring
Introduction: Hi, We're running a enterprise SaaS solution that currently holds about 700 customers with up to 50.000 users per customer (growing quickly). Our customers have SLA agreements with us that contains guaranteed uptimes, response times and other performance counters. With an increasing number of customers and traffic we find it difficult to provide our customer with actual SLA data. We could set up external probes that monitors certain parts of the application, but this is time consuming with 700 customers (we do it today for our biggest clients). We can also extract data from web logs but they are now approaching about 30-40 GB a day. What we really need is monitoring software that not only focuses on the internal performance counters but also lets us see the application from the customers viewpoint and allows us to aggregate data in different ways. Would the best approach be to develop a custom solution (for instance a distributed app that aggregates data from different logs e
3 0.7168594 1160 high scalability-2011-12-21-In Memory Data Grid Technologies
Introduction: After winning a CSC Leading Edge Forum (LEF) research grant, I (Paul Colmer) wanted to publish some of the highlights of my research to share with the wider technology community. What is an In Memory Data Grid? It is not an in-memory relational database, a NOSQL database or a relational database. It is a different breed of software datastore. In summary an IMDG is an ‘off the shelf’ software product that exhibits the following characteristics: The data model is distributed across many servers in a single location or across multiple locations. This distribution is known as a data fabric. This distributed model is known as a ‘shared nothing’ architecture. All servers can be active in each site. All data is stored in the RAM of the servers. Servers can be added or removed non-disruptively, to increase the amount of RAM available. The data model is non-relational and is object-based. Distributed applications written on the .NET and Java application platforms are s
Introduction: Much of the focus of high performance computing (HPC) has centered on CPU performance. However, as computing requirements grow, HPC clusters are demanding higher rates of aggregate data throughput. Today's clusters feature larger numbers of nodes with increased compute speeds. The higher clock rates and operations per clock cycle create increased demand for local data on each node. In addition, InfiniBand and other high-speed, low-latency interconnects increase the data throughput available to each node. Traditional shared file systems such as NFS have not been able to scale to meet this growing demand for data throughput on HPC clusters. Scalable cluster file systems that can provide parallel data access to hundreds of nodes and petabytes of storage are needed to provide the high data throughput required by large HPC applications, including manufacturing, electronic design, and research. This paper describes an implementation of the Sun Lustre file system as a scalable storage
5 0.70928091 42 high scalability-2007-07-30-Product: GridLayer. Utility computing for online application
Introduction: TGL delivers Virtual Private Datacenters and virtual private servers from grids of commodity servers. Each TGL grid consists of a pool of HP servers connected with a Gigabit backbone network and running 3Tera's AppLogic grid operating system. With a Virtual Private Datacenter, you get complete control of your own private grid. Using our visual interface, you set up and assemble disposable virtual infrastructure, including firewalls, load balancers, web servers, database servers, NAS boxes, etc. visually, by pointing and clicking. You can build advanced clusters, deploy large and small applications, and save them as templates that can be provisioned in minutes. You can even build your own virtual servers and appliances.
6 0.70659137 827 high scalability-2010-05-14-Hot Scalability Links for May 14, 2010
7 0.70387661 292 high scalability-2008-03-30-Scaling Out MySQL
8 0.69534165 504 high scalability-2009-01-29-Event: MySQL Conference & Expo 2009
9 0.69313151 30 high scalability-2007-07-26-Product: AWStats a Log Analyzer
10 0.6877501 310 high scalability-2008-04-29-High performance file server
11 0.68531203 924 high scalability-2010-10-21-What is Network-based Application Virtualization and Why Do You Need It?
12 0.68529928 486 high scalability-2009-01-07-Sun Acquires Q-layer in Cloud Computing Play
13 0.68458331 1082 high scalability-2011-07-18-New Relic Architecture - Collecting 20+ Billion Metrics a Day
14 0.68416589 1220 high scalability-2012-04-02-YouPorn - Targeting 200 Million Views a Day and Beyond
15 0.68190205 652 high scalability-2009-07-08-Art of Parallelism presentation
16 0.68098271 970 high scalability-2011-01-06-BankSimple Mini-Architecture - Using a Next Generation Toolchain
17 0.68070537 1472 high scalability-2013-06-07-Stuff The Internet Says On Scalability For June 7, 2013
18 0.67986327 271 high scalability-2008-03-08-Product: DRBD - Distributed Replicated Block Device
19 0.67983526 319 high scalability-2008-05-14-Scaling an image upload service
20 0.67978579 366 high scalability-2008-08-17-Many updates against MySQL