high_scalability high_scalability-2007 high_scalability-2007-145 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Hi, I would like feed back on a ID generator I just made. What positive and negative effects do you see with this. It's programmed in Java, but could just as easily be programmed in any other typical language. It's thread safe and does not use any synchronization. When testing it on my laptop, I was able to generate 10 million IDs within about 15 seconds, so it should be more than fast enough. Take a look at the attachment.. (had to rename it from IdGen.java to IdGen.txt to attach it) IdGen.java
sentIndex sentText sentNum sentScore
1 Hi, I would like feed back on a ID generator I just made. [sent-1, score-0.612]
2 What positive and negative effects do you see with this. [sent-2, score-0.722]
3 It's programmed in Java, but could just as easily be programmed in any other typical language. [sent-3, score-1.254]
4 It's thread safe and does not use any synchronization. [sent-4, score-0.358]
5 When testing it on my laptop, I was able to generate 10 million IDs within about 15 seconds, so it should be more than fast enough. [sent-5, score-0.597]
wordName wordTfidf (topN-words)
[('programmed', 0.479), ('rename', 0.334), ('attach', 0.273), ('generator', 0.269), ('laptop', 0.257), ('negative', 0.233), ('positive', 0.226), ('ids', 0.216), ('effects', 0.204), ('safe', 0.196), ('feed', 0.18), ('generate', 0.162), ('seconds', 0.145), ('id', 0.138), ('typical', 0.13), ('thread', 0.129), ('testing', 0.113), ('easily', 0.107), ('java', 0.094), ('within', 0.094), ('back', 0.084), ('million', 0.081), ('able', 0.08), ('fast', 0.067), ('look', 0.062), ('could', 0.059), ('see', 0.059), ('would', 0.048), ('use', 0.033), ('like', 0.031)]
simIndex simValue blogId blogTitle
same-blog 1 1.0 145 high scalability-2007-11-08-ID generator
Introduction: Hi, I would like feed back on a ID generator I just made. What positive and negative effects do you see with this. It's programmed in Java, but could just as easily be programmed in any other typical language. It's thread safe and does not use any synchronization. When testing it on my laptop, I was able to generate 10 million IDs within about 15 seconds, so it should be more than fast enough. Take a look at the attachment.. (had to rename it from IdGen.java to IdGen.txt to attach it) IdGen.java
2 0.10870203 606 high scalability-2009-05-25-non-sequential, unique identifier, strategy question
Introduction: (Please bare with me, I'm a new, passionate, confident and terrified programmer :D ) Background: I'm pre-launch and 1 year into the development of my application. My target is to be able to eventually handle millions of registered users with 5-10% of them concurrent. Up to this point I've used auto-increment to assign unique identifiers to rows. I am now considering switching to a non-sequential strategy. Oh, I'm using the LAMP configuration. My reasons for avoiding auto-increment: 1. Complicates replication when scaling horizontally. Risk of collision is significant (when running multiple masters). Note: I've read the other entries in this forum that relate to ID generation and there have been some great suggestions -- including a strategy that uses auto-increment in a way that avoids this pitfall... That said, I'm still nervous about it. 2. Potential bottleneck when retrieving/assigning IDs -- IDs assigned at the database. My reasons for being nervous about
3 0.099057496 1538 high scalability-2013-10-28-Design Decisions for Scaling Your High Traffic Feeds
Introduction: Guest post by Thierry Schellenbach, Founder/CTO of Fashiolista.com, follow @tschellenbach on Twitter and Github Fashiolista started out as a hobby project which we built on the side. We had absolutely no idea it would grow into one of the largest online fashion communities. The entire first version took about two weeks to develop and our feed implementation was dead simple. We’ve come a long way since then and I’d like to share our experience with scaling feed systems. Feeds are a core component of many large startups such as Pinterest, Instagram, Wanelo and Fashiolista. At Fashiolista the feed system powers the flat feed , aggregated feed and the notification system. This article will explain the troubles we ran into when scaling our feeds and the design decisions involved with building your own solution. Understanding the basics of how these feed systems work is essential as more and more applications rely on them. Furthermore we’ve open sourced Feedly , the Python m
4 0.096448876 1303 high scalability-2012-08-13-Ask HighScalability: Facing scaling issues with news feeds on Redis. Any advice?
Introduction: We just released a social section to our iOS app several days ago and we are already facing scaling issues with the users' news feeds. We're basically using a Fan-out-on-write (push) model for the users' news feeds (posts of people and topics they follow) and we're using Redis for this (backend is Rails on Heroku). However, our current 60,000 news feeds is ballooning our Redis store to almost 1GB in a just a few days (it's growing way too fast for our budget). Currently we're storing the entire news feed for the user (post id, post text, author, icon url, etc) and we cap the entries to 300 per feed. I'm wondering if we need to just store the post IDs of each user feed in Redis and then store the rest of the post information somewhere else? Would love some feedback here. In this case, our iOS app would make an api call to our Rails app to retrieve a user's news feed. Rails app would retrieve news feed list (just post IDs) from Redis, and then Rails app would need to query to g
5 0.090572834 346 high scalability-2008-06-28-ID generation schemes
Introduction: Hi, Generating unique ids is a common requirements in many projects. Generally, this responsibility is given to Database layer. By using sequences or some other technique. This is a problem for horizontal scalability. What are the Guid generation schemes used in high scalable web sites generally? I have seen use java's SecureRandom class to generate Guid. What are the other methods generally used? Thanks Unmesh
6 0.074486189 788 high scalability-2010-03-04-How MySpace Tested Their Live Site with 1 Million Concurrent Users
8 0.067227863 407 high scalability-2008-10-10-The Art of Capacity Planning: Scaling Web Resources
9 0.067064144 659 high scalability-2009-07-20-A Scalability Lament
10 0.066730045 459 high scalability-2008-12-03-Java World Interview on Scalability and Other Java Scalability Secrets
11 0.065584347 1302 high scalability-2012-08-10-Stuff The Internet Says On Scalability For August 10, 2012
12 0.06545455 141 high scalability-2007-11-05-Quick question about efficiently implementing Facebook 'news feed' like functionality
13 0.0641983 1440 high scalability-2013-04-15-Scaling Pinterest - From 0 to 10s of Billions of Page Views a Month in Two Years
14 0.061046042 1040 high scalability-2011-05-13-Stuff The Internet Says On Scalability For May 13, 2011
15 0.056471825 187 high scalability-2007-12-14-The Current Pros and Cons List for SimpleDB
16 0.055604365 561 high scalability-2009-04-08-N+1+caching is ok?
17 0.05498239 430 high scalability-2008-10-26-Should you use a SAN to scale your architecture?
18 0.053311288 1008 high scalability-2011-03-22-Facebook's New Realtime Analytics System: HBase to Process 20 Billion Events Per Day
19 0.053051721 1038 high scalability-2011-05-11-Troubleshooting response time problems – why you cannot trust your system metrics
20 0.052599113 1456 high scalability-2013-05-13-The Secret to 10 Million Concurrent Connections -The Kernel is the Problem, Not the Solution
topicId topicWeight
[(0, 0.062), (1, 0.03), (2, -0.021), (3, -0.015), (4, 0.021), (5, -0.021), (6, 0.014), (7, 0.036), (8, 0.003), (9, -0.012), (10, 0.011), (11, 0.018), (12, 0.021), (13, -0.006), (14, -0.023), (15, -0.018), (16, -0.01), (17, -0.033), (18, 0.004), (19, 0.002), (20, -0.017), (21, -0.059), (22, -0.009), (23, -0.015), (24, 0.001), (25, -0.03), (26, 0.009), (27, 0.003), (28, 0.025), (29, -0.015), (30, 0.012), (31, -0.035), (32, -0.021), (33, -0.039), (34, 0.004), (35, -0.01), (36, -0.01), (37, 0.01), (38, -0.011), (39, 0.029), (40, -0.008), (41, 0.002), (42, -0.015), (43, 0.023), (44, 0.005), (45, 0.025), (46, 0.004), (47, -0.015), (48, -0.035), (49, 0.011)]
simIndex simValue blogId blogTitle
same-blog 1 0.94066918 145 high scalability-2007-11-08-ID generator
Introduction: Hi, I would like feed back on a ID generator I just made. What positive and negative effects do you see with this. It's programmed in Java, but could just as easily be programmed in any other typical language. It's thread safe and does not use any synchronization. When testing it on my laptop, I was able to generate 10 million IDs within about 15 seconds, so it should be more than fast enough. Take a look at the attachment.. (had to rename it from IdGen.java to IdGen.txt to attach it) IdGen.java
Introduction: Here's an Update On Disqus: It's Still About Realtime, But Go Demolishes Python . How do you add realtime functionality to a web scale application? That's what Adam Hitchcock , a Software Engineer at Disqus talks about in an excellent talk: Making DISQUS Realtime ( slides ). Disqus had to take their commenting system and add realtime capabilities to it. Not something that's easy to do when at the time of the talk (2013) they had had just hit a billion unique visitors a month. What Disqus developed is a realtime commenting system called “realertime” that was tested to handle 1.5 million concurrently connected users, 45,000 new connections per second, 165,000 messages/second, with less than .2 seconds latency end-to-end. The nature of a commenting system is that it is IO bound and has a high fanout, that is a comment comes in and must be sent out to a lot of readers. It's a problem very similar to what Twitter must solve . Disqus' solution was quite interesting as was th
3 0.59934402 1538 high scalability-2013-10-28-Design Decisions for Scaling Your High Traffic Feeds
Introduction: Guest post by Thierry Schellenbach, Founder/CTO of Fashiolista.com, follow @tschellenbach on Twitter and Github Fashiolista started out as a hobby project which we built on the side. We had absolutely no idea it would grow into one of the largest online fashion communities. The entire first version took about two weeks to develop and our feed implementation was dead simple. We’ve come a long way since then and I’d like to share our experience with scaling feed systems. Feeds are a core component of many large startups such as Pinterest, Instagram, Wanelo and Fashiolista. At Fashiolista the feed system powers the flat feed , aggregated feed and the notification system. This article will explain the troubles we ran into when scaling our feeds and the design decisions involved with building your own solution. Understanding the basics of how these feed systems work is essential as more and more applications rely on them. Furthermore we’ve open sourced Feedly , the Python m
4 0.59278941 1074 high scalability-2011-07-06-11 Common Web Use Cases Solved in Redis
Introduction: In How to take advantage of Redis just adding it to your stack Salvatore 'antirez' Sanfilippo shows how to solve some common problems in Redis by taking advantage of its unique data structure handling capabilities. Common Redis primitives like LPUSH, and LTRIM, and LREM are used to accomplish tasks programmers need to get done, but that can be hard or slow in more traditional stores. A very useful and practical article. How would you accomplish these tasks in your framework? Show latest items listings in your home page . This is a live in-memory cache and is very fast. LPUSH is used to insert a content ID at the head of the list stored at a key. LTRIM is used to limit the number of items in the list to 5000. If the user needs to page beyond this cache only then are they sent to the database. Deletion and filtering . If a cached article is deleted it can be removed from the cache using LREM . Leaderboards and related problems . A leader board is a set sorted by score.
5 0.58476841 1644 high scalability-2014-05-07-Update on Disqus: It's Still About Realtime, But Go Demolishes Python
Introduction: Our last article on Disqus: How Disqus Went Realtime With 165K Messages Per Second And Less Than .2 Seconds Latency , was a little out of date, but the folks at Disqus have been busy implementing, not talking, so we don't know a lot about what they are doing now, but we do have a short update in C1MM and NGINX by John Watson and an article Trying out this Go thing . So Disqus has grown a bit: 1.3 billion unique visitors 10 billion page views 500 million users engaged in discussions 3 million communities 25 million comments They are still all about realtime, but Go replaced Python in their Realtime system: Original Realtime backend was written in a pretty lightweight Python + gevent. The realtime service is a hybrid of CPU intensive tasks + lots of network IO. Gevent was handling the network IO without an issue, but at higher contention, the CPU was choking everything. Switching over to Go removed that contention, which was the primary issue that was being se
7 0.5792076 1454 high scalability-2013-05-08-Typesafe Interview: Scala + Akka is an IaaS for Your Process Architecture
8 0.56962895 1158 high scalability-2011-12-16-Stuff The Internet Says On Scalability For December 16, 2011
9 0.56682205 1269 high scalability-2012-06-20-iDoneThis - Scaling an Email-based App from Scratch
10 0.55534261 317 high scalability-2008-05-10-Hitting 300 SimbleDB Requests Per Second on a Small EC2 Instance
11 0.55360746 307 high scalability-2008-04-21-Using Google AppEngine for a Little Micro-Scalability
12 0.5497759 1507 high scalability-2013-08-26-Reddit: Lessons Learned from Mistakes Made Scaling to 1 Billion Pageviews a Month
13 0.54440171 639 high scalability-2009-06-27-Scaling Twitter: Making Twitter 10000 Percent Faster
14 0.54342347 1609 high scalability-2014-03-11-Building a Social Music Service Using AWS, Scala, Akka, Play, MongoDB, and Elasticsearch
15 0.54320347 1340 high scalability-2012-10-15-Simpler, Cheaper, Faster: Playtomic's Move from .NET to Node and Heroku
16 0.54007155 1573 high scalability-2014-01-06-How HipChat Stores and Indexes Billions of Messages Using ElasticSearch and Redis
17 0.53801274 1108 high scalability-2011-08-31-Pud is the Anti-Stack - Windows, CFML, Dropbox, Xeround, JungleDisk, ELB
18 0.53744048 550 high scalability-2009-03-30-Ebay history and architecture
19 0.53363597 545 high scalability-2009-03-19-Product: Redis - Not Just Another Key-Value Store
20 0.52953494 711 high scalability-2009-09-22-How Ravelry Scales to 10 Million Requests Using Rails
topicId topicWeight
[(1, 0.077), (2, 0.198), (46, 0.437), (61, 0.117)]
simIndex simValue blogId blogTitle
1 0.82861406 95 high scalability-2007-09-17-Scalable CMS?
Introduction: What do you guys think/know about the scalability of the popular CMSs (like Joomla, Drupal or Typo3)? Any experience/suggestions there? I'm not sure which to pick yet... Thanks, Stephan
same-blog 2 0.82297885 145 high scalability-2007-11-08-ID generator
Introduction: Hi, I would like feed back on a ID generator I just made. What positive and negative effects do you see with this. It's programmed in Java, but could just as easily be programmed in any other typical language. It's thread safe and does not use any synchronization. When testing it on my laptop, I was able to generate 10 million IDs within about 15 seconds, so it should be more than fast enough. Take a look at the attachment.. (had to rename it from IdGen.java to IdGen.txt to attach it) IdGen.java
3 0.82012326 894 high scalability-2010-09-03-Six guiding principles to Consolidate your IT
Introduction: The need for IT consolidation is most evident in two types of organizations. In the first group, IT grew organically with business over the decades, and survived changes of strategy, management, staff and vendor orientation. The second group of businesses capital groups are characterized by rapid growth through acquisitions (followed by attempts to integrate radically different IT environments). In both groups, their IT infrastructures have typically been pieced together over the past 20 (or more) years.ďťż Read more on BigDataMatters.com
4 0.74473369 634 high scalability-2009-06-20-Building a data cycle at LinkedIn with Hadoop and Project Voldemort
Introduction: Update : Building Voldemort read-only stores with Hadoop . A write up on what LinkedIn is doing to integrate large offline Hadoop data processing jobs with a fast, distributed online key-value storage system, Project Voldemort .
5 0.67729378 597 high scalability-2009-05-12-GemStone Unveils GemFire Enterprise 6.0
Introduction: GemFire Enterprise is in-memory distributed data management platform that pools memory (and CPU, network and optionally local disk) across multiple processes to manage application objects and behavior. With the 6.0 release, GemFire has reached a stage of maturity in its evolution. GemStone touts this version as the true 'best of breed' distributed caching technology, solving scalability issues in all industries.
6 0.62701356 335 high scalability-2008-05-30-Is "Scaling Engineer" a new job title?
7 0.57732308 41 high scalability-2007-07-30-Product: Flickr
8 0.55794555 1486 high scalability-2013-07-03-5 Rockin' Tips for Scaling PHP to 30,000 Concurrent Users Per Server
9 0.55330902 1464 high scalability-2013-05-24-Stuff The Internet Says On Scalability For May 24, 2013
10 0.54325122 1154 high scalability-2011-12-09-Stuff The Internet Says On Scalability For December 9, 2011
11 0.54273582 541 high scalability-2009-03-16-Product: Smart Inspect
12 0.53041089 703 high scalability-2009-09-12-How Google Taught Me to Cache and Cash-In
13 0.52233893 53 high scalability-2007-08-01-Product: MogileFS
14 0.50965869 1325 high scalability-2012-09-19-The 4 Building Blocks of Architecting Systems for Scale
15 0.48825234 288 high scalability-2008-03-25-Paper: On Designing and Deploying Internet-Scale Services
16 0.47722825 685 high scalability-2009-08-20-Dependency Injection and AOP frameworks for .NET
17 0.47708505 877 high scalability-2010-08-12-Designing Web Applications for Scalability
18 0.46592355 1135 high scalability-2011-10-31-15 Ways to Make Your Application Feel More Responsive under Google App Engine
19 0.46425962 564 high scalability-2009-04-10-counting # of views, calculating most-least viewed
20 0.46394899 80 high scalability-2007-09-06-Product: Perdition Mail Retrieval Proxy