Posted On: Friday, 26 December 2008 by Rajiv Popat

When I first figured out how Resume Driven Development works I naively assumed that I could blame it for every project that is mesh of super complex technologies and tricky code. After experiencing it for the first time, I assumed that for every project that used countless layers of abstractions, indirections, irrelevant technologies held together using really weak band-aids, there had to be one or more team members desperately trying to build their resume or drooling over the latest cutting edge technology and frameworks that are in Beta.

It was then that I met a different kind of programmers. These could not be defined as full fledged proponents of Resume Driven Development, yet it was the kind whose mere involvement in any projects could turn their projects into what we shall refer to as ‘Project Rocket Science’:

These are programmers  who make a serious face and talk about ‘Enterprise Development’, ‘Best Practices’ and ‘Scalability’. When I say ‘talk’ I’m not referring to the ‘bring-up-in-a-white-board-session-and-brain-storm-quickly’ kind of a talk. I’m talking about start-a-formal-document and discuss-for-a-few-weeks-or-maybe-even-months kind of talk. Or for that matter make-a-big-deal-about-it and don't-start-coding-till-you’ve-figured-it-out kind of talk. After all, this thing is really important and these individuals just ‘have to’ get it right before they start coding.

At Multiplitaxion Inc, we had a hard time convincing a few of these programmers to try out Ruby on Rails. Before individuals wrote a single line of ruby code discussions and arguments pertaining to ‘is-ruby-on-rails-scalable’ began. The discussions and arguments lasted for weeks. Examples of ‘enterprise applications’ written in Ruby On Rails were demanded and provided in long mail trails. Days went by; weeks later we were moving in circles trying to know for sure we can write ‘Enterprise Applications’ using Ruby On Rails.

Damien Katz, has an interesting take on zealotry and excessive concern about scalability and building ‘enterprise level application’. Damien believes that if you find yourself making a big deal about some of these topics, chances are that you're a crappy programmer and you don’t even know it:

You know those crappy programmers who don’t know they are crappy? You know, they think they're pretty good, they spout off the same catch phrase rhetoric they've heard some guru say and they know lots of rules about the "correct" way to do things? Yet their own work seems seriously lacking given all the expertise they supposedly have? You don’t know any programmers like that? Come one, you know, the guys who are big on dogma but short on understanding. No, doesn’t sound familiar?

Then here are some signs you're a crappy programmer and don't know it:

Java is all you'll ever need: You don't see the need for other languages, why can't everything be in Java? It doesn't bother you at all to see Python or Ruby code that accomplishes in 10 lines what takes several pages in Java. Besides, you're convinced new language features in the next release will fix all that anyway.(BTW, this can be almost any language, but right now the Java community seems most afflicted with this thinking)

"Enterprise" isn't a punchline to you: This is serious stuff dammit. "Enterprise" is not just a word, it's a philosophy, a way of life, a path to enlightenment. Anything that can be written, deployed or upgraded with minimal fuss is dismissed as a toy that won't "scale" for future needs. Meanwhile most of the real work in your office is getting done by people sending around Excel spreadsheets as they wait for your grand enterprise visions to be built. 

It’s a rather wordy and highly opinionated post. I encourage all you Java developers to flame Damien (not me) for all of it. On a serious note, without being specific about any language or platform, the whole Enterprise-and-Scalability-Talk is highly overrated in a lot of development communities around the world.

What is even more frustrating is that the people talking about these topics the most are the shittiest programmers, incapable of creating applications that can support a trivial twenty concurrent users. Ted Dziuba feels so utterly frustrated about people clueless about scalability making a big deal about scalability that he loses a track of language in his rather foul mouthed yet passionate, hilarious and honest post:

Engineers love to talk about scalability.  It makes us feel like the bad ass, [swearword]-swingin' [swearword] that we wish we could be.

After we talk about scalability with our co-workers (Yeah, Rails doesn't scale!), we flex our true engineering prowess by writing a post about it on our blog.  Once that post hits Reddit, son, everyone will know how hardcore you really are.  Respect.

People Who Talk Big About Scalability Don't Need To Worry About It.

Fact:  every chest-thumping blog post I have seen written about scalability is either about architecture, Memcached, or both.  Some a#@hole who writes shitty code starts pontificating about "scalable architecture" with data storage, web frontends, whatever-the-f#@k.  Dude, your app isn't having scalability problems because of the architecture.  It's having scalability problems because you coded a ton of N^2 loops into it and you're too self-important to get peer reviews on your commits.

And let's not forget the tools who discover Memcached for the first time, install it on a web server, and notice how fast their app runs now.  Yeah, welcome to the modern age.  Hope you know what a cache expiry policy is.

If You Haven't Discussed Capacity Planning, You Can't Discuss Scalability.

You don't need to worry about scalability on your Rails-over-Mysql application because nobody is going to use it.  Really.  Believe me.  You're going to get, at most, 1,000 people on your app, and maybe 1% of them will be 7-day active.  Scalability is not your problem, getting people to give a shit is. 

The folks at 37Sginals are much more conservative with their language; but they make the exact same point that getting-people-to-give-a-shit is indeed your biggest problem, with a rather nicely worded and elegant chapter in their book. They explain:

"Will my app scale when millions of people start using it?"

Ya know what? Wait until that actually happens. If you've got a huge number of people overloading your system then huzzah! That's one swell problem to have. The truth is the overwhelming majority of web apps are never going to reach that stage. And even if you do start to get overloaded it's usually not an all-or-nothing issue. You'll have time to adjust and respond to the problem. Plus, you'll have more real-world data and benchmarks after you launch which you can use to figure out the areas that need to be addressed. 

I’ve dealt with a few performance issues myself. I’ve also seen a few kick ass programmers kick some serious butt by tweaking the database to it’s limit and take caching to it’s limit. What’s fundamentally different was that in all these cases, we invariably waited till it became a real problem. When it did become a real problem we got into a room, white boarded the issue and got it fixed without making a big noise about it.

I’ve done this with two different teams in over three projects and the one conclusion on scalability that I’ve reached is this; if ‘Enterprise’ and ‘Scalability’ are big words in your organization you’re definitely doing something wrong; or maybe you just have too many individuals in your team who cannot write code or ship.

User your common sense, avoid un-necessary complexity and write simple code. Chances are you won’t even run into scalability problems till your application grows to a level where you’ll actually love having and fixing scalability issues.

The next time you talk about setting up a cluster for load balancing, building an enterprise application or utter the ‘S’ word, show me a million users signed up for your application or a ‘real scalability problem in production’ and we can talk ‘scalability’. Till you can do that, just keep those N^2 loops or that funny-little-short-cut out of your code and you’ll be good.

You can flex your engineering muscle and continue whining about Scalability, Best Practices and ‘Enterprise’ applications all you want. In the meanwhile we’re going to write some shitty software with bugs, get something done, and ship stuff that people like using. You know what; we’ll even have fun doing it. I’m not all all that into talking about ‘Enterprise Application’. I don’t give a shit about ‘Scalability’ till it becomes a real problem; and believe me that’s not as bad as it sounds.


Comment Section

Comments are closed.