nathan_marz_storm nathan_marz_storm-2013 nathan_marz_storm-2013-35 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Yesterday was my last day at Twitter. I left to start my own company. What I'll be working on is very exciting (though I'm keeping it secret for now). Leaving Twitter was a tough decision. I worked with a whole bunch of great people on fascinating problems with some of the most interesting data in the world. Ultimately though, I felt that if I didn't make this move, I would regret it for the rest of my life. So I put in my papers about a month ago and then spent a month transitioning my team for my departure. This ends an eventful three years that started with me joining BackType in January of 2010. So much has happened in these past three years. I open-sourced Cascalog , ElephantDB , and Storm , started writing a book , gave a lot of talks , and in July of 2011 experienced the thrill of being acquired . My projects spread beyond BackType and Twitter to be relied on by dozens and dozens of companies . Through all this, I learned an enormous amount about entrepreneurship, p
sentIndex sentText sentNum sentScore
1 What I'll be working on is very exciting (though I'm keeping it secret for now). [sent-3, score-0.464]
2 I worked with a whole bunch of great people on fascinating problems with some of the most interesting data in the world. [sent-5, score-0.545]
3 Ultimately though, I felt that if I didn't make this move, I would regret it for the rest of my life. [sent-6, score-0.287]
4 So I put in my papers about a month ago and then spent a month transitioning my team for my departure. [sent-7, score-1.076]
5 This ends an eventful three years that started with me joining BackType in January of 2010. [sent-8, score-0.807]
6 I open-sourced Cascalog , ElephantDB , and Storm , started writing a book , gave a lot of talks , and in July of 2011 experienced the thrill of being acquired . [sent-10, score-0.598]
7 My projects spread beyond BackType and Twitter to be relied on by dozens and dozens of companies . [sent-11, score-0.88]
8 Through all this, I learned an enormous amount about entrepreneurship, product development, marketing, recruiting, and project management. [sent-12, score-0.481]
wordName wordTfidf (topN-words)
[('month', 0.245), ('three', 0.245), ('backtype', 0.19), ('started', 0.176), ('entrepreneurship', 0.176), ('ends', 0.176), ('papers', 0.176), ('relied', 0.176), ('marketing', 0.176), ('january', 0.176), ('exciting', 0.176), ('dozens', 0.176), ('secret', 0.176), ('tough', 0.176), ('twitter', 0.153), ('bunch', 0.152), ('enormous', 0.152), ('felt', 0.152), ('experienced', 0.152), ('worked', 0.152), ('rest', 0.135), ('beyond', 0.135), ('ultimately', 0.135), ('spread', 0.135), ('move', 0.135), ('talks', 0.135), ('book', 0.135), ('spent', 0.122), ('joining', 0.122), ('ago', 0.122), ('though', 0.117), ('happened', 0.112), ('keeping', 0.112), ('left', 0.112), ('recruiting', 0.112), ('last', 0.103), ('whole', 0.103), ('learned', 0.103), ('past', 0.095), ('team', 0.095), ('years', 0.088), ('project', 0.082), ('projects', 0.082), ('development', 0.082), ('day', 0.082), ('product', 0.077), ('put', 0.071), ('problems', 0.071), ('amount', 0.067), ('great', 0.067)]
simIndex simValue blogId blogTitle
same-blog 1 0.9999997 35 nathan marz storm-2013-03-16-Leaving Twitter
Introduction: Yesterday was my last day at Twitter. I left to start my own company. What I'll be working on is very exciting (though I'm keeping it secret for now). Leaving Twitter was a tough decision. I worked with a whole bunch of great people on fascinating problems with some of the most interesting data in the world. Ultimately though, I felt that if I didn't make this move, I would regret it for the rest of my life. So I put in my papers about a month ago and then spent a month transitioning my team for my departure. This ends an eventful three years that started with me joining BackType in January of 2010. So much has happened in these past three years. I open-sourced Cascalog , ElephantDB , and Storm , started writing a book , gave a lot of talks , and in July of 2011 experienced the thrill of being acquired . My projects spread beyond BackType and Twitter to be relied on by dozens and dozens of companies . Through all this, I learned an enormous amount about entrepreneurship, p
2 0.13823035 39 nathan marz storm-2014-02-12-Interview with "Programmer Magazine"
Introduction: I was recently interviewed for "Programmer Magazine", a Chinese magazine. The interview was published in Chinese, but a lot of people told me they'd like to see the English version of the interview. Due to the Google translation being, ahem, a little iffy, I decided to just publish the original English version on my blog. Hope you enjoy! What drew you to programming and what was the first interesting program you wrote? I started programming when I was 10 years old on my TI-82 graphing calculator. Initially I started programming because I wanted to make games on my calculator – and also because I was bored in math class :D. The first interesting game I made on my calculator was an archery game where you'd shoot arrows at moving targets. You'd get points for hitting more targets or completing all the targets faster. A couple years later I graduated to programming the TI-89 which was a huge upgrade in power. I remember how the TI-82 only let you have 26 variables (for the character
3 0.11019871 30 nathan marz storm-2011-03-29-My talks at POSSCON
Introduction: Last week I went to POSSCON in Columbia, South Carolina. It was an interesting experience and a good reminder that not everyone in the world thinks like we do in Silicon Valley. I gave two talks at the conference. One was a technical talk about how to build realtime Big Data systems, and the other was a non-technical talk about the things we do at BackType to be a super-productive team. Both slide decks are embedded below. The Secrets of Building Realtime Big Data Systems Become Efficient or Die: The Story of BackType
4 0.082557142 19 nathan marz storm-2010-07-12-My experience as the first employee of a Y Combinator startup
Introduction: I'm the first employee of BackType , a Summer '08 YC company. My joining the company increased the company size by 50%. The experience has been awesome, but I will say up front that being the first employee of a startup is not for everyone. The best part of being the first employee of a startup is the total exposure to all parts of the company. I've learned a ton about product development, customer development, recruiting, and entrepreneurship. Additionally, I've met and connected with lots of other awesome people through the YC network. I've gotten all these benefits at relatively low risk for myself, as I still have a salary and a solid chunk of equity. No Rules There are a lot of rules working at most companies. You don't even realize that some of the rules are rules until you work at a company with no rules. I'm talking about the most basic things like what hours you work, what days of the week you work, what tools you use, and whether you come into the office or not. B
5 0.078658685 32 nathan marz storm-2012-01-09-Early access edition of my book is available
Introduction: The early access edition of my book Big Data: principles and best practices of scalable realtime data systems is now available from Manning! I've been working on this book for quite some time, and I'm excited to have it out there and start getting some feedback. The interest in the book has already been overwhelming, and I've been answering questions about it on Hacker News .
6 0.074391335 34 nathan marz storm-2012-09-19-Storm's 1st birthday
7 0.061924387 4 nathan marz storm-2010-01-26-My conversation with the great John McCarthy
8 0.058192369 38 nathan marz storm-2013-04-12-Break into Silicon Valley with a blog
9 0.054830957 22 nathan marz storm-2010-10-05-How to get a job at a kick-ass startup (for programmers)
10 0.0531107 18 nathan marz storm-2010-06-16-Your company has a knowledge debt problem
11 0.051751811 5 nathan marz storm-2010-01-30-Stateless, fault-tolerant scheduling using randomness
12 0.050961807 25 nathan marz storm-2010-12-06-You Are a Product
13 0.050845221 27 nathan marz storm-2011-01-07-Analysis of the #LessAmbitiousMovies Twitter Meme
14 0.05056734 40 nathan marz storm-2014-02-24-The inexplicable rise of open floor plans in tech companies
15 0.047625456 7 nathan marz storm-2010-03-04-Introducing "Nanny" - a really simple dependency management tool
16 0.047542308 28 nathan marz storm-2011-01-11-Cascalog workshop
17 0.037588771 23 nathan marz storm-2010-10-27-Fastest Viable Product: Investing in Speed at a Startup
18 0.036051348 41 nathan marz storm-2014-05-10-Why we in tech must support Lawrence Lessig
19 0.034843482 33 nathan marz storm-2012-02-06-Suffering-oriented programming
20 0.031471185 3 nathan marz storm-2010-01-13-Mimi Silbert: the greatest hacker in the world
topicId topicWeight
[(0, 0.23), (1, 0.111), (2, -0.046), (3, 0.121), (4, 0.222), (5, -0.241), (6, -0.121), (7, 0.094), (8, 0.048), (9, 0.018), (10, 0.092), (11, 0.147), (12, 0.083), (13, 0.034), (14, 0.362), (15, -0.124), (16, 0.238), (17, -0.116), (18, 0.349), (19, 0.04), (20, -0.197), (21, 0.036), (22, -0.162), (23, -0.143), (24, -0.219), (25, 0.219), (26, 0.17), (27, -0.095), (28, 0.029), (29, 0.31), (30, 0.064), (31, -0.169), (32, 0.059), (33, -0.051), (34, -0.191), (35, -0.066), (36, 0.062), (37, -0.011), (38, 0.07), (39, 0.002), (40, 0.003)]
simIndex simValue blogId blogTitle
same-blog 1 0.9957754 35 nathan marz storm-2013-03-16-Leaving Twitter
Introduction: Yesterday was my last day at Twitter. I left to start my own company. What I'll be working on is very exciting (though I'm keeping it secret for now). Leaving Twitter was a tough decision. I worked with a whole bunch of great people on fascinating problems with some of the most interesting data in the world. Ultimately though, I felt that if I didn't make this move, I would regret it for the rest of my life. So I put in my papers about a month ago and then spent a month transitioning my team for my departure. This ends an eventful three years that started with me joining BackType in January of 2010. So much has happened in these past three years. I open-sourced Cascalog , ElephantDB , and Storm , started writing a book , gave a lot of talks , and in July of 2011 experienced the thrill of being acquired . My projects spread beyond BackType and Twitter to be relied on by dozens and dozens of companies . Through all this, I learned an enormous amount about entrepreneurship, p
2 0.16676697 39 nathan marz storm-2014-02-12-Interview with "Programmer Magazine"
Introduction: I was recently interviewed for "Programmer Magazine", a Chinese magazine. The interview was published in Chinese, but a lot of people told me they'd like to see the English version of the interview. Due to the Google translation being, ahem, a little iffy, I decided to just publish the original English version on my blog. Hope you enjoy! What drew you to programming and what was the first interesting program you wrote? I started programming when I was 10 years old on my TI-82 graphing calculator. Initially I started programming because I wanted to make games on my calculator – and also because I was bored in math class :D. The first interesting game I made on my calculator was an archery game where you'd shoot arrows at moving targets. You'd get points for hitting more targets or completing all the targets faster. A couple years later I graduated to programming the TI-89 which was a huge upgrade in power. I remember how the TI-82 only let you have 26 variables (for the character
3 0.12569872 19 nathan marz storm-2010-07-12-My experience as the first employee of a Y Combinator startup
Introduction: I'm the first employee of BackType , a Summer '08 YC company. My joining the company increased the company size by 50%. The experience has been awesome, but I will say up front that being the first employee of a startup is not for everyone. The best part of being the first employee of a startup is the total exposure to all parts of the company. I've learned a ton about product development, customer development, recruiting, and entrepreneurship. Additionally, I've met and connected with lots of other awesome people through the YC network. I've gotten all these benefits at relatively low risk for myself, as I still have a salary and a solid chunk of equity. No Rules There are a lot of rules working at most companies. You don't even realize that some of the rules are rules until you work at a company with no rules. I'm talking about the most basic things like what hours you work, what days of the week you work, what tools you use, and whether you come into the office or not. B
4 0.12178159 30 nathan marz storm-2011-03-29-My talks at POSSCON
Introduction: Last week I went to POSSCON in Columbia, South Carolina. It was an interesting experience and a good reminder that not everyone in the world thinks like we do in Silicon Valley. I gave two talks at the conference. One was a technical talk about how to build realtime Big Data systems, and the other was a non-technical talk about the things we do at BackType to be a super-productive team. Both slide decks are embedded below. The Secrets of Building Realtime Big Data Systems Become Efficient or Die: The Story of BackType
5 0.098423667 32 nathan marz storm-2012-01-09-Early access edition of my book is available
Introduction: The early access edition of my book Big Data: principles and best practices of scalable realtime data systems is now available from Manning! I've been working on this book for quite some time, and I'm excited to have it out there and start getting some feedback. The interest in the book has already been overwhelming, and I've been answering questions about it on Hacker News .
6 0.094956763 38 nathan marz storm-2013-04-12-Break into Silicon Valley with a blog
7 0.092118114 27 nathan marz storm-2011-01-07-Analysis of the #LessAmbitiousMovies Twitter Meme
8 0.089839675 22 nathan marz storm-2010-10-05-How to get a job at a kick-ass startup (for programmers)
9 0.080225311 25 nathan marz storm-2010-12-06-You Are a Product
10 0.079539321 34 nathan marz storm-2012-09-19-Storm's 1st birthday
11 0.077635437 40 nathan marz storm-2014-02-24-The inexplicable rise of open floor plans in tech companies
12 0.072812796 4 nathan marz storm-2010-01-26-My conversation with the great John McCarthy
13 0.072550714 18 nathan marz storm-2010-06-16-Your company has a knowledge debt problem
14 0.072303176 23 nathan marz storm-2010-10-27-Fastest Viable Product: Investing in Speed at a Startup
15 0.068630919 7 nathan marz storm-2010-03-04-Introducing "Nanny" - a really simple dependency management tool
16 0.063841283 5 nathan marz storm-2010-01-30-Stateless, fault-tolerant scheduling using randomness
17 0.061082207 31 nathan marz storm-2011-10-13-How to beat the CAP theorem
18 0.060873564 3 nathan marz storm-2010-01-13-Mimi Silbert: the greatest hacker in the world
19 0.059952535 41 nathan marz storm-2014-05-10-Why we in tech must support Lawrence Lessig
20 0.057154901 33 nathan marz storm-2012-02-06-Suffering-oriented programming
topicId topicWeight
[(41, 0.026), (76, 0.848)]
simIndex simValue blogId blogTitle
1 0.99957854 32 nathan marz storm-2012-01-09-Early access edition of my book is available
Introduction: The early access edition of my book Big Data: principles and best practices of scalable realtime data systems is now available from Manning! I've been working on this book for quite some time, and I'm excited to have it out there and start getting some feedback. The interest in the book has already been overwhelming, and I've been answering questions about it on Hacker News .
same-blog 2 0.99625409 35 nathan marz storm-2013-03-16-Leaving Twitter
Introduction: Yesterday was my last day at Twitter. I left to start my own company. What I'll be working on is very exciting (though I'm keeping it secret for now). Leaving Twitter was a tough decision. I worked with a whole bunch of great people on fascinating problems with some of the most interesting data in the world. Ultimately though, I felt that if I didn't make this move, I would regret it for the rest of my life. So I put in my papers about a month ago and then spent a month transitioning my team for my departure. This ends an eventful three years that started with me joining BackType in January of 2010. So much has happened in these past three years. I open-sourced Cascalog , ElephantDB , and Storm , started writing a book , gave a lot of talks , and in July of 2011 experienced the thrill of being acquired . My projects spread beyond BackType and Twitter to be relied on by dozens and dozens of companies . Through all this, I learned an enormous amount about entrepreneurship, p
3 0.23607914 8 nathan marz storm-2010-03-08-Follow-up to "The mathematics behind Hadoop-based systems"
Introduction: In a previous post , I developed an equation modeling the stable runtime of an iterative, batch-oriented workflow. We saw how the equation explained a number of counter-intuitive behaviors of batch-oriented systems. In this post, we will learn how to measure the amount of overhead versus dynamic time in a workflow, which is the first step in applying the theory to optimize a workflow. Recall that we started with the equation for the runtime of a single iteration of a workflow: Runtime = Overhead + {Time to process one hour of data} * {Hours of Data} T = O + P * H We ended with the equation for the stable runtime of a workflow that runs repeatedly: {Stable Runtime} = Overhead / (1 - {Time to process one hour of data} T = O / (1 - P) Measuring O and P The first step towards utilizing this theory for optimizing your workflow will be to measure the values of O and P for your workflow. This can be difficult if the cluster is shared with lots of other jobs, as the P for each ru
4 0.23272212 39 nathan marz storm-2014-02-12-Interview with "Programmer Magazine"
Introduction: I was recently interviewed for "Programmer Magazine", a Chinese magazine. The interview was published in Chinese, but a lot of people told me they'd like to see the English version of the interview. Due to the Google translation being, ahem, a little iffy, I decided to just publish the original English version on my blog. Hope you enjoy! What drew you to programming and what was the first interesting program you wrote? I started programming when I was 10 years old on my TI-82 graphing calculator. Initially I started programming because I wanted to make games on my calculator – and also because I was bored in math class :D. The first interesting game I made on my calculator was an archery game where you'd shoot arrows at moving targets. You'd get points for hitting more targets or completing all the targets faster. A couple years later I graduated to programming the TI-89 which was a huge upgrade in power. I remember how the TI-82 only let you have 26 variables (for the character
5 0.21764839 1 nathan marz storm-2009-12-28-The mathematics behind Hadoop-based systems
Introduction: I wish I had known this a year ago. Now, with some simple mathematics I can finally answer: Why doesn't the speed of my workflow double when I double the amount of processing power? Why does a 10% failure rate cause my runtime to go up by 300%? How does optimizing out 30% of my workflow runtime cause the runtime to decrease by 80%? How many machines should I have in my cluster to be adequately performant and fault-tolerant? All of these questions are neatly answered by one simple equation: Runtime = Overhead / (1 - {Time to process one hour of data}) We will derive this equation in a moment. First, let's briefly discuss what I mean by "Hadoop-based system." 1 A common use-case of Hadoop is running a workflow that processes a continuous stream of incoming data. The workflow runs in a "while(true)" loop, and each iteration of the workflow processes the data that accumulated since last iteration. The inspiration for the following analysis can be summarized in a simp
6 0.20595971 25 nathan marz storm-2010-12-06-You Are a Product
7 0.19403605 18 nathan marz storm-2010-06-16-Your company has a knowledge debt problem
8 0.19333376 7 nathan marz storm-2010-03-04-Introducing "Nanny" - a really simple dependency management tool
9 0.19213617 27 nathan marz storm-2011-01-07-Analysis of the #LessAmbitiousMovies Twitter Meme
10 0.16239232 9 nathan marz storm-2010-03-10-Thrift + Graphs = Strong, flexible schemas on Hadoop
11 0.15194508 38 nathan marz storm-2013-04-12-Break into Silicon Valley with a blog
12 0.14117138 23 nathan marz storm-2010-10-27-Fastest Viable Product: Investing in Speed at a Startup
13 0.13733476 40 nathan marz storm-2014-02-24-The inexplicable rise of open floor plans in tech companies
14 0.1340683 19 nathan marz storm-2010-07-12-My experience as the first employee of a Y Combinator startup
15 0.12091874 22 nathan marz storm-2010-10-05-How to get a job at a kick-ass startup (for programmers)
16 0.11265135 11 nathan marz storm-2010-03-23-Migrating data from a SQL database to Hadoop
17 0.11150168 17 nathan marz storm-2010-05-26-Why your company should have a very permissive open source policy
18 0.099978805 28 nathan marz storm-2011-01-11-Cascalog workshop
19 0.097103417 24 nathan marz storm-2010-11-03-The time I hacked my high school
20 0.096385807 16 nathan marz storm-2010-05-08-News Feed in 38 lines of code using Cascalog