free html hit counter
Posted on: Friday, April 13, 2007 by Rajiv Popat

The Thick-Skinned Developer

Every now and then I meet young, talented and enterprising developers and interns who are hesitant to post code snippets, out of their work environment into the open world. There are some who will not write Code-Project articles for the fear of getting negative feedbacks.

Others won’t speak at knowledge sharing sessions or trainings, even if it’s just within their own organization, until they are fully prepared with an elaborate PPT and have spent hours learning a topic and know it like they know their mother-tongue. Some even worry too much about what their audience thought about a particular training they conducted last month.

Here’s a piece of advice to every young intern and engineer starting out on their careers:

Learn how to get naked!

You should have nothing to hide! Nothing. If you are succeeding in your project, say it. If you are failing, say it in a louder voice! If you are an expert in a topic, say it. If you don’t know a thing about a topic, say it, in a louder voice!

And once you’re naked, develop a thicker skin!

Don’t let the critisms get you down. Instead, use them to become a better developer (and a better person).

Being a think-skinned shameless developer who has nothing to hide is a life-style. After you have lived it, you begin to wonder how the developers, who don’t follow it, live without it :)

Are you a thick-skinned developer? Are you ready to become one? Try the life-style! You might love it! Here are a few steps that can help get you started:

  1. Realize how much your code sucks - and then learn to say it with a sense of humor - shamelessly. After all, we all make shitty software… with Bugs
  2. Ask for more time and make your code better - Pick one of your modules that sucks the most and ask for some extra time to re-factor it. If it’s not ready to ship and you need to work harder, learn to say so without feeling particularly ashamed about it
  3. Besides writing code learn to talk about code - Try walking into a training that you are about to conduct with just a topic you think you know really well or feel passionately about. No elaborate PPTs, No Preparations! If possible, do this every few weeks. 
  4. Throw out a few articles at code-project or start a blog - Blogs and articles open up forms of dialogs where it’s really easy to be shameless and ask questions or say “I don’t know” while answering them. The critism that comes in as comments through your blogs, articles or just ad-hoc email from your readers is usually very constructive. Personally I have and will (hopefully) continue to, learn a lot from Ad-hoc emails and feedback I receive from this blog and my articles.  
  5. Read Code – Read code that’s better than code that you would generally write – Quite a few good open source projects are usually a great place to start. If you can’t commit to an open source project submit bug-fixes and patches for the ones you use in your every-day life! 
  6. Copy Shamelessly – I’ve copied patterns of coding from every good programmer I ever worked with. In my professional life at work, the most recent example of shameless copy was a different way of writing Configuration Handlers that I picked up from a very capable Developer in my current project.

The above points don't form a definitive guide to becoming a shameless thick-skinned developer but if you’ve never made fun of your own code in a meeting or in a public blog post – they might help you get started on the road of thick-skinned-shamelessness where you might be able to learn much more by being a little more shameless and thick-skinned! I wish you a Safe and a Happy Journey on this road of shameless and thick-skinned development.

posted on Friday, April 13, 2007 8:55:29 AM UTC by Rajiv Popat  #    Comments [4]
Posted on: Thursday, March 29, 2007 by Rajiv Popat

Some Gangi Talk from the Past

I think it’s been more than a year since this incident happened. Which is when I had  written this and shared it out with selected friends and acquaintances on a collective blog I shared with friends at that time. Recently I was reminded by someone that I should probably post this out on my blog :)

For all the readers who are not familiar with Indian languages here are some words used in the story and what they mean.

“Jadu” is a special kind of a broom usually kept at Indian house-holds. Besides sweeping the floor this thing is supposed to have multiple other uses, like…

Hitting cockroaches hard enough to leave them unconscious, without killing them! :)

“Gangi” is Hindi-Language-Equivalent for a vest. So here’s the post - back from the old-days. Read on!

Some More Gangi Talk from Yesterday…

Yesterday was “sofa king” (to be read very fast without stopping) long and interesting - thought I should share it out for the sake of some creative speculations that you guys might have.

A Little background: Tuesday evening I see a big fat cockroach crawling on the walls of the bathroom of my hotel suite. That is when I realized that hotels in Texas don’t give a jadu with rooms… I missed my jadu then.

Anyways, I decided to use a "gangi", climbed up the bathtub caught the bugger walking high on the wall and left him on the stairs of the hotel so that he could conveniently go into someone else’s suite and bother them. Thought that was the end of it… turns out, I was “sofa king” wrong!

Yesterday morning, just when I was about to leave for office, I see a cockroach (the same one? Milan, a friend back from school days, suspects that it was a relative of the one thrown out on Tuesday, but I have my doubts – I mean, personally, I am inclined to believe now that with the years of evolution behind them these guys can easily find their way back to the bathroom of the suite from which they were insulted and thrown out! Can’t they?)

So I see him walking on the same high spot on the same wall! (I think he was back for revenge of the insult he had been subjected to a couple of days ago) Anyways, now the turn of events begin…

The plan was laid out – the primary weapon of attack that was so successful last time (the same gangi) will be used again and the rascal will be thrown right on the streets of Houston! So, I take the gangi and climb the bathtub again! Just as I was about the attack, the rascal decides to fly right on my shirt!!! Well, I did what anyone who hasn’t participated in fear factor would do – decided to RUN! But… I slipped from the bathtub instead.

In a desperate attempt to hold my ground I tried to catch hold of the bathroom curtain rod – which I thought, being made of solid steel, would be able hold my weight! Again, I was “sofa king” wrong!

The rod just decides to break into two pieces (that was a steel rod! I am not kidding here – albeit, it was hollow but it was steel! Makes me wonder if I am putting on weight or as ifte (another friend from school days) says, becoming - 'muscular' – Well, it breaks into two pieces, comes completely off the wall, throwing the curtain on the ground. This time, the cockroach decides to hide in the kitchen area! Right under a cabin!

Now my mind is multitasking with the primary thread being – Who the phu##$% pays for this?? The secondary threads all occupied by the cockroach who was still sitting there in the kitchen!

So, I decided to go to the front desk and tell them about the cockroaches! I thought everything would be fine now. Well again, I need not say by now that I was so… wrong!

Basically, I go to the front desk with all the 3 key cards of my room safely locked in the room!

So I go and tell them – “Houston, there’s a problem” – and the lady out there starts filing a ticket for me assuring me that they will “look into it”. Then I broke the news that I have already wreaked their bathroom rod while I myself was trying to “look into it”.

Now I am taken a bit seriously. A guy is sent with me to the room. When I ask for an extra key I am told that I’ve already used up the keys the hotel had for my room – but the guy can open the door for me. Good! I think… once I get in I can get all my keys.

On the way up (we took the elevator to go to the second floor) I have a hard time describing to the guy what a cockroach is. Not sure why – but looks like the guy has never heard the word cockroach before (or maybe I am just not articulate enough). Interesting.

So he finally opens my door for me using his master key (or whatever). Luckily for me the bugger is still sitting in my kitchen! Jackpot! My point is proved! I point at the bugger and tell him – “these” (whatever you want to call them) have been coming to my room for the past 2 days!! Now he gets it or in his words – he sees what I mean. He bashes the bugger with his handkerchief and throws it out.

He tells me that there’s been some renovation going on and they’re spraying insecticides around the building so cockroaches are moving in. At the end of the day...?

I get all my three keys, I go to office… some really exciting work going on these days which makes professional life fun (but better to leave the technical mumbo jumbo out) … and when I am back I can see my room all cleaned up with the insecticide sprayed in the entire suite! And the broken rod with the curtain… it’s still lying under my sink.

posted on Thursday, March 29, 2007 9:45:09 AM UTC by Rajiv Popat  #    Comments [0]
Posted on: Saturday, March 3, 2007 by Rajiv Popat

Are You Stuck With a Problem? Divide and Conquer!

At a client’s office, almost a year ago, a very talented developer on their team thought it was interestingly funny when I said - “and that’s how you divide and rule”. “We call it Divide and conquer” – he laughed. [Being an Indian, at times my English tends to have a slightly different touch to it. :)]

“But you’re probably correct. You don’t just want to conquer. What you really want to do is to conquer and then control the situation and rule, for a long time.” - He added on.

No, we were not making a sinister plan to take over the world. We were just talking about how we can break one big Monolithic BizTalk Orchestration into smaller, easy to mange, easy to comprehend and faster orchestrations.

He had drawn up this Big Design of their existing approach on the left and then I had proceeded to draw smaller designs of multiple Orchestrations and how they would talk to each other on the right when I finally wrapped up the meeting with the line “divide and rule”. I was suggesting that he break his Orchestrations down to smaller moving parts so that when a part doesn't work - he can take it out - independently - and Analyze it!

I don't think any programmer disagrees with the fact that we should Code Smaller. A Lot of basic concept articles, teach a person who is starting out on his programming endeavors how he can Break Larger Pieces of Code into Smaller Pieces. But divide and conquer (/rule) design paradigm goes beyond just writing cohesive, small and easy to maintain functions and classes.

Every now and then I see even good developers forget the Divide and Conquer paradigm under pressure. Or they simply do not put it to practice for tasks other than coding. I've recently seen a very capable business analyst trying to learn build management by taking a CuriseControl configuration file and trying to configure CriuseControl to get the source code from SVN, fire the build, send email notification when the build fails and log the result in the formatted HTML report, all at once.

When I was called in to help, my first reaction was to wipe out all other code from the 50+ line config file - and leave just 5 lines that will have cruise control fire the build. Turns out, those five lines were broken. Once that was fixed we moved on to Getting the source code from SVN - another 3 lines which just worked. We kept fixing a part at a time and soon we were done in a couple of hours.

I've seen so many engineers (Highly capable ones. At times, I'm guilty of this too) run through more than 10 screens before they can reach a reported bug and just reproduce it. Then apply the fix, go through the screens all over again only to find out that the fix doesn't work. Then start thinking of a different fix. Over and Over again. Spending more than 5 minutes on every attempt, just to check if the fix worked. 

When we do this the time taken to fix a bug increases dramatically, the frustration level increases dramatically and the ability to think using a Pragmatic Approach comes down with every failed attempt. We start programming by co-incidence! In situations like these, Isolating the problem by replicating it using smaller code snippets and moving it out of your codebase so that you can analyze it separately, can help a lot. 

If you are trying to fix a bug that requires more than a couple of minutes to just reproduce and has taken more than 5 attempted fixes already, you might be better off isolating the bug by bringing it out of the codebase and reproducing it independently using simple code snippets. If you are finding a bug too complex to fix you're probably trying to fix more than one bugs at a time. If a feature you're building in your sprint seems too complicated you're probably trying to build two connected features simultaneously.

In the pressure of fixing a bug that refuses to get fixed or in the excitement of writing code for something fairly complex, it's very easy to tell yourself - "I'm almost there!" - and it's easy to spend away hours or sometime even days - by continuing to tell yourself that, not realizing that you're in fact, getting nowhere!

If something takes much longer to fix or build than you initially expected or you're starting to get frustrated with a problem - Divide and Conquer! And then, once you've conquered the problem, enjoy your Rule over the situation! :) 

posted on Saturday, March 3, 2007 7:44:47 PM UTC by Rajiv Popat  #    Comments [0]
Posted on: Tuesday, February 13, 2007 by Rajiv Popat

Reading XML Files with PowerShell (Monad)

A couple of months ago I had to make a difficult choice. A choice between Vista and Powershell (I still need time getting used to this name:)). Vista had officially RTM'ed and Powershell for Vista wasn't available. It was a difficult choice - wait on XP/2003 and continue to use Powershell or Move to Vista and wait for Powershell to become Vista Ready. I decided to move to Vista.

Recently, after about a couple of months, exactly as promised by Jeffrey Snover, the Moand team has Vista'ed the Powershell Installer and that means I can have both Vista and Powershell now. To add to that, I can totally continue to Skin Powershell and make it look beautiful, just like my Vista. And that's very important! Right? :)

I've been playing a lot with Powershell and now I think it's time for me, to put it to some real use, in project deployments on QA / Production.

One of the problems I always had while writing anything about PowerShell is that there are just too many good things to talk about in there and almost anything that could have been written about, has been written by someone else. That's one of the reasons (excuses? :)) why I haven't been writing a lot about it. For now, I'm just going to start by talking a little bit about XML and how you can read XML files in Powershell.

Let's just assume that we have the following simple XML file which lets me run various Uninstall and Install SQL scripts on the database depending on the Application version that I am pushing to the server.


<DBScripts>
<version name="1.0.0">
<script UnInstallScript="uninstall1-1.sql" InstallScript="install1-1.sql"/>
<script UnInstallScript="uninstall1-2.sql" InstallScript="install1-2.sql"/>
</version>
<version name="2.0.0">
<script UnInstallScript="uninstall2-1.sql" InstallScript="install2-1.sql"/>
<script UnInstallScript="uninstall2-2.sql" InstallScript="install2-2.sql"/>
</version>
</DBScripts>
 

Typically, creating a database project, a command file and then a custom Console application in C# that reads this XML file and fires corresponding SQL Scripts might work, but it takes some time. Let's use Powershell to see how easy it is to read this XML.

Let's start with simplest approach first. XML support is pretty neatly baked into Powershell. So assuming this file was called Mockfile.xml, I can just create a XmlDocument object using the following command at my Powershell window:


PS E:\> [xml] $MyXmlDocument = Get-Content MockFile.xml
 

That's it. The XMLDocument is now available in this instance variable. Let's take a look at what the variable $MyXmlDocuent contains:


PS E:\> Get-Variable MyXmlDocument

NameValue Value
---- -----
MyXmlDocument #document

PS E:\> $MyXmlDocument

DBScripts
---------
DBScripts
 

We're all set to read from the document now. Let's try to get to various nodes on the XML File. Because, everything returned by Powershell is an object, to get the value of the Node all I Need to do is access the attribute within the object. For Example, to get all values of version nodes inside the the DBScripts Node and store it inside a separate variable I could just do:


PS E:\> $MyVersions = $MyXmlDocument.DBScripts
PS E:\> $MyVersions

version
-------
{1.0.0, 2.0.0}
 

Notice how I access the DBScripts Node and assign all version nodes inside the root node to a different variable. To see value value of $MyVersions variable that I just created I just type it in and Powershell confirms that it now has nodes pertaining to both versions in this newly created variable.

Now let's assume I need to reach the version node with the Name 2.0.0. and access all the script files in there. To do this let's quickly take a look at the version node collection which was returned by the above command:


PS E:\> $MyVersions.version

name script
---- ------
1.0.0 {script, script}
2.0.0 {script, script}

Now to directly access the version node where the name is 2.0.0, let's use PowerShell filtering and let's pipe the output of version nodes to the the Where-Object commandlet:


PS E:\> $SecondVersion = $MyVersions.version | where-object {$_.name -match '2.0.0'}
PS E:\> $SecondVersion

name script
---- ------
2.0.0 {script, script}

PS E:\> $SecondVersion.script

UnInstallScript InstallScript
-------------- -------------
uninstall2-1.sql install2-1.sql
uninstall2-2.sql install2-2.sql

Once I've piped the output of all my version attribute to the where-object commandlet and assigned the final output to another variable I inspect the attribute the variable contains by typing $SecondVersion and then peek into it's script collection by doing $SecondVersion.script. If this was a deployment script, the above command would have given me all the scripts I need to run to install the version 2.0.0 of my application. I could easily convert this to a Powershell script where the version number comes in a parameter. The script can go ahead and figure out which SQL script it needs to run and then can run them. However, the focus of this Post is to concentrate of reading XML files, so let's try to do everything we just did using just two lines of code:


PS E:\> [xml] $MyXmlDocument = Get-Content MockFile.xml
PS E:\> ($MyXmlDocument.DBScripts.version | where-object {$_.name -match '2.0.0'}).script

UnInstallScript InstallScript
-------------- -------------
uninstall2-1.sql install2-1.sql
uninstall2-2.sql install2-2.sql

Sweet! Don't you think? :) Wait, there's more! You could actually do this using a single command at the command prompt if you wanted to:


PS E:\> (([xml] (Get-Content MockFile.xml)).DBScripts.version | Where-Object {$_.name -match '2.0.0'}).script

UnInstallScript Installscript
-------------- -------------
uninstall2-1.sql install2-1.sql
uninstall2-2.sql install2-2.sql

And then of course you could iterate through this list by using the ForEach-Object commandlet if you wanted to:


PS E:\> (([xml] (Get-Content MockFile.xml)).DBScripts.version | Where-Object {$_.name -match '2.0.0'}).script | ForEach-
Object { "Here I Could Run : " + $_.InstallScript }

Here I Could Run : install2-1.sql
Here I Could Run : install2-2.sql

PS E:\>
 

Or you could just access a particular element by it's zero based index:


PS E:\> (([xml] (Get-Content MockFile.xml)).DBScripts.version | Where-Object {$_.name -match '2.0.0'}).script[1]

UnInstallScript InstallScript
-------------- -------------
uninstall2-2.sql install2-2.sql

PS E:\>

And did I mention that two code snippets above are just one line commands? You could of-course turn this into a script depending on what you are trying to do. But what if you were not very comfortable with this approach and wanted to use familiar XPath to access XML nodes? Don't run away! You can do that too. This MSDN article shows you how.

Here's wishing you Happy-XML with Powershell!

posted on Tuesday, February 13, 2007 1:21:20 PM UTC by Rajiv Popat  #    Comments [0]
Posted on: Sunday, February 11, 2007 by Rajiv Popat

The Art of Throwing Away Code

Andrew Hunt and Venkat Subramaniam use the Turkish proverb, “No matter how far you’ve gone down the wrong road you’ve gone, turn back.”, rather elegantly in their book “Practices of an Agile Developer”. They comment:

"That Turkish proverb above is both simple and obvious—you’d think it would be a guiding force for software development. But all too often, developers (including your humble authors) continue down the wrong road in the misguided hope that it will be OK somehow. Maybe it’s close enough. Maybe this isn’t really as wrong a road as it feels. We might even get away with it now and then, if creating software were a linear, deterministic process—like the proverbial road. But it’s not."

 

 

Software development is about making choices. Starting from selecting the underlying frameworks your code will run on, tools you will use, a host of other choices and above all, the assumptions your code will make.

We make choices at every step. While we try our best to make our choices based on objective and pragmatic thinking, the fact is that our choices are based on the facts and requirements available at any given time. And of-course, in a real world, these change. Very often. Then, there is the pressure to ship and every developer needs to compromise some level of perfection for the sake of shipping!

Making just a couple of wrong choices can put you on the wrong way. You can try your best to avoid getting on the wrong way. I wish you luck. But the truth is, no matter how hard you try, you will take a couple of wrong turns once in a while. Building software is like driving on a confusing mesh of freeways, with an outdated map, that changes every five minutes, and having only a very general idea of where you want to get to. At some point or the other, chances are that you will take a couple of wrong exits and will get lost.

In a recent mail trail team members of a project and me discussed fixing 3 bugs in first few iterations of module. The module, which was a very small module of a large web based product, consisted of just 3 aspx pages. While these discussions were continuing, I received an email from a person who was assigned the responsibility of fixing these 3 bugs. He had completed his initial analysis on what it would take to fix those 3 bugs and this email contained his suggestion on how these bugs can be fixed. The email contained just a single line of text – “I think the code needs to be re-written.”

I consider emails of this sort particularly helpful. They tell me that we’re failing fast (And for those who didn’t click that link, that’s a good thing :)). Emails of this sort, tell me that even before we go to Testing, members in the team have actually figured out that we’ve just taken a wrong way and have raised the “wrong way” sign:

That just makes turning back very easy.

In this particular case, after a quick follow-up meeting on the mail-trail everyone agreed that we needed to fix a couple of bugs on a page, refactor a page ruthlessly and rewrite just one page because re-writing it would be faster than fixing it and would make things much better.

While spending hours and hours in trying to put band-aids on code that doesn’t work, or is badly designed, at rare occasions it’s a good idea to start fresh. In other words, “turn back”.

While turning back, works on micro-level modules, classes, small functions or code snippets, at a macro-level / project-level, more often than not, turning back is not a viable option.

At a product / project level, Joel Spolsky feels this is one of those “Things you should never do”. He explains:

"We're programmers. Programmers are, in their hearts, architects, and the first thing they want to do when they get to a site is to bulldoze the place flat and build something grand. We're not excited by incremental renovation: tinkering, improving, planting flower beds.

There's a subtle reason that programmers always want to throw away the code and start over. The reason is that they think the old code is a mess. And here is the interesting observation: they are probably wrong. The reason that they think the old code is a mess is because of a cardinal, fundamental law of programming:

It's harder to read code than to write it"

He goes ahead to explain why taking a huge project and attempting to rewrite it from scratch is not such a great idea:

"The idea that new code is better than old is patently absurd. Old code has been used. It has been tested. Lots of bugs have been found, and they've been fixed. There's nothing wrong with it. It doesn't acquire bugs just by sitting around on your hard drive."

But then, how do you turn back, especially if you have made too many bad choices and your project is a huge one? A Wise comment on this forum explains how: 

"Get together a refactoring plan. This shows how you can go from a mess to something useable in incremental steps.

I've never seen a project where absolutely every single line of code was a waste. Even if the architecture underneath is flawed, the UI might be okay. So keep the UI and refactor what goes on underneath.

It's always going to be quicker than a rewrite."

Pick and choose what you need to throw away, when you need to throw it away and how much of it do you really need to throw away. While you do that, don’t forget to keep the risks-of-throwing-away-code to a minimum. When you can throw away small snippets and fix things by rigorous yet phased Refactoring, don’t attempt a monolithic rewrite by throwing away all your code.

Where am I trying to get with this? If you haven’t already noticed, I’m just trying to refactor the proverb I started off with. So, here’s my refactored version: 

“No Matter how far down the wrong road you’ve gone, find the shortest, safest and smartest way to the right road.” - If writing code is an art, throwing it away is an art too! 

posted on Sunday, February 11, 2007 7:12:45 PM UTC by Rajiv Popat  #    Comments [0]
Posted on: Tuesday, February 6, 2007 by Rajiv Popat

Fixing Package Load Failure Error with Windows Workflow Foundation and Visual Studio

I've often said that the really good part about Windows is What can be done, or sometimes... what happens ("seemingly automatically"), can be undone (and explained). The same is also true for Visual Studio 2005. At rare occasions there are errors in the IDE which seem magical and mysterious. With the right debugging techniques, commands, some time the mysterious errors can be fixed and explained.

I've been busily coding away at Windows Workflow Foundation for a few months now. Currently I'm using the final released version of Windows workflow foundation with the released version of Workflow extensions for Visual Studio, and my experience with WF has been very good. So when this strange error popped up today without any particular reason, I was a little surprised.

The error wasn't very helpful in describing what was going wrong. All it stated was: Package Load Failure. It basically said - "Package 'Microsoft.Workflow.VSDesginer.DesignerPackage, Microsoft.Workflow.VSDesigner, version=3.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' has failed to Load Properly (GUID = {some_guid}..." and then it went on to ask me If I wanted to disable my workflow foundation extensions for now.

Even though my workflow project opened and worked perfectly this was just an irritating error which popped up every time I started Visual Studio 2005. Determined to put an end to this harmless yet irritating error, I started my investigation and Google searches.

I started off with basic commands like - "devenv.exe /setup" - which did not fix the issue at hand. Then there were posts out there which suggested that I delete my native images and regenerate them using the ngen.exe tool. That solution did not seem to work either.

After a few hours of Google searches I came across this post which basically described what the issue was. Turns out, a few days ago I had tried my hand at writing Visual Studio Add-ins and had forgotten to delete an Add-In I had written. Remains of my HelloWorld Addin existed in "C:\Users\admin\Documents\Visual Studio 2005\Addins" on my Vista Partition.

All I had to do was to delete the ".Addin" file (which btw, had nothing to do with Windows Workflow Foundation, so it's a little difficult to suspect this would be the cause of the problem) and as soon as I had done that Visual Studio Started without any Windows Workflow Extension Add-in errors.

If this doesn't do the trick for you - here are few other commands / resources which are usually helpful in catching and fixing Visual Studio.NET 2005 IDE issues:

  1. devenv.exe /setup - very useful for a host of other IDE setting related problems or if you just want to restore VS settings to their original state. Did not help in my case because the failed add-in was something I had written myself and my problem had nothing to do with visual studio settings. In my case, the settings were just fine, even though I had initially suspected that they had gone corrupt. 
  2. ngen.exe - Aaron Stebner's Post describes this pretty elaborately. The basic idea here is to delete your Visual Studio Native Image files and regenerate them using this tool.
  3. devenv.exe /log - this generates XML log which can be quite helpful in trouble-shooting various startup issues with the IDE.

And if you're still struggling Aaron Stebner also provides a Definitive list of workarounds for Package Load Failure errors in Visual Studio 2005, here.

We spend so much of our time with the IDE everyday writing code. Mysterious errors like give us a chance to look under the hood and see what's going on. It's just like the joy of fixing that little problem in your car, yourself. After all knowing how to fix minor problems with your car, or knowing how to replace a flat tire, is also important besides knowing how to drive. Isn't it? :)

posted on Tuesday, February 6, 2007 6:02:34 PM UTC by Rajiv Popat  #    Comments [0]
Posted on: Monday, February 5, 2007 by Rajiv Popat

Why Does Fred Program By Coincidence

Andy Hunt and David Thomas in their book “The Pragmatic Programmer” make an interesting observation about a persona Fred:

"Suppose Fred is given a programming assignment. Fred types in some code, tries it, and it seems to work. Fred types in some more code, tries it, and it still seems to work. After several weeks of coding this way, the program suddenly stops working, and after hours of trying to fix it, he still doesn't know why. Fred may well spend a significant amount of time chasing this piece of code around without ever being able to fix it. No matter what he does, it just doesn't ever seem to work right.

Fred doesn't know why the code is failing because he didn't know why it worked in the first place. It seemed to work, given the limited "testing'' that Fred did, but that was just a coincidence. Buoyed by false confidence, Fred charged ahead into oblivion. Now, most intelligent people may know someone like Fred, but we know better. We don't rely on coincidences---do we?

Sometimes we might. Sometimes it can be pretty easy to confuse a happy coincidence with a purposeful plan"

During a discussion with one of my early project Managers, a Mentor and a person I really look up to, he coined Fred’s symptoms and gave it an appropriate name – he calls this the “Junior Developer Syndrome” and feels that every programmer was once Fred.

He also describes another very important feature of Fred. According to him (I couldn’t agree more), during these “hours of trying to fix it”, this Fred guy usually doesn’t tell anyone there’s a problem.

Like all other programmers the first couple of years of my coding life went in getting rid of the Fred within me. These years also taught me lessons all of us have learnt at point in our life. This post is an attempt to organize my own thoughts and try and understand why Fred programs by co-incidence.

There are lots of other complex factors that go into software development life; however, to keep this post simple let’s consider that Fred is worried about the tree most evident factors in a software development project.

  1. Quality of Code.
  2. Time Needed to write Code.
  3. Pressure to ship (from Stake-holders).

I’ll just go ahead and use graphs to illustrate how these three typically play together.

The Graph Plots Time needed to code against the quality of code. Spending a little bit of more time in understanding the Technology and figuring out the “why the code works” part gives a tremendous boost to the quality of code by investing just a little bit of time. The initial tweaks and optimizations are easily achieved by spending a little bit of time on the technology and understanding why the code works and how it works. The Graph depicts this as the distance between points A and B. i.e. [AB].

The most typical example of this I’ve seen is ASP.NET caching. For example, I’ve been on projects where a 4 second Page load was brought down to 300 milliseconds by just spending a day on Caching. In other words spending [AB] time gave us a [XY] increase in the quality of code.

After you’ve spent that initial time however, the law of Diminishing returns starts to kick in and every other tweak on code takes more and more time. In the same project I mentioned earlier we spent over a week of performance tuning to bring the 300 milliseconds down to less than 150. In other words, we ended up spending [BC] time to increase the quality of code by [YZ].

Now the obvious question is if by spending just [AB] worth of time learning about the “why” and the “how things works under the hood” of the technology on which Fred is working on, if Fred can get and [XY] gain in the quality of code why doesn’t he invest that time in making his code better?

In this past couple of years I've discussed this (in one way or the other) to multiple Senior Associates, Technical Architects and Project Managers. Some have told me that Fred is just stupid. Others claim that Fred just works for a Pay-check and [OX] worth of quality is usually good enough to get the project go through client acceptance. Fred couldn’t care less about code.

Others however, mentioned a reason that sounds more likely – The third factor I talked about earlier in the post - Pressure to ship – that is what causes Fred “Program by Co-incidence”.

Notice the graph above. Here the pressure to ship show up suddenly as your product / project starts getting shape or starts generating excitement. As you move from point E to F tweaking your code, the pressure to ship slowly growing up by [PQ]. As you move from F to G and the Shipping-Date / Dead-Line / Committed Date comes closer the pressure to deliver shoots hugely by [QR].

Now let’s merge both of these graphs.

Merging these two graphs answers some of the questions and gives a new perspective into – Why does Fred write Bad code. Think about it a little more and it makes so much more sense.

The problem with Fred is that he considers his code ready to ship as soon as he senses some pressure to Ship from Stake-holders. He considers his code ready-to-ship at point X. Fred wants to see the Stake-holders happy. He’s not a bad developer! Or, is he?

The other extreme are people who will not ship till point Z. People who will spend weeks to fine-tune a tiny performance issue that is causing a round-trip to take 15 milliseconds more than what it should take. This kind of a developer runs the risk of Never Shipping, because the Green curve never becomes parallel to the X-axis. In other words, no matter how much you tune and tweak your code there is always room for improvement.

So when should you ship?

During one of my first projects in my development life (when I was very young and enthusiastic :)) I shipped a project at point X and paid the price for it. The code I shipped very soon became my problem; within a couple of months I saw myself taking support calls and fixing bugs by working over 18 hours a day; hating the code I had written in a hurry and fixing it where possible. It could have bitten me really bad, but thank god it didn’t. We fixed all major issues that were reported and No-one was killed. There was nothing about that project I enjoyed in particular, but it taught me to strike a balance. It taught me to ship at Point M because M is an idle point to ship.

Shipping at point M in the above graph, involves it's own set of challenges though. It means maintaining constant communication with the Stake-holders, reporting accurate status, remaining agile and selling the idea that Quality would mean lesser cost in the long run to the Stake-holders even at the time when pressure to ship is constantly increasing.

What does this boil down to? While the book suggests some excellent code-related-techniques of how Fred can become a programmer who does not program by coincidence, in my opinion however, It also boils down the fact that to ship at M Fred needs to learn a whole host of other skills which have nothing to do with coding. Skills which Fred always considered unimportant as a developer. A mentor remarks - "Software development is not all about For loops. A lot of software development is about Managing Risks and so many other things". These are In fact, words of wisdom which deserve a different post for themselves.

posted on Monday, February 5, 2007 12:05:43 PM UTC by Rajiv Popat  #    Comments [1]
Posted on: Friday, January 26, 2007 by Rajiv Popat

The When and Why of SVN Branches, Tags and Trunk

At work we use SVN for all our .NET Projects. Being a Microsoft Developer I’ve moved between multiple source control systems during my development life. Starting with Visual Source Safe, I moved to CVS, then to TFS (during its Beta stages) and finally fell in love and settled down with SVN. The love was strong enough to convert all .NET developers I work with, over to SVN! :)

Even today I highly encourage all Microsoft developers I meet outside the workplace to try-out SVN. With Tortoise and Ankh, the whole Visual Studio Integration and Explorer Integration is very easy to get and it (almost) feels as easy as VSS and as powerful as CVS. But this post is not about plugging-in SVN and related tools. It’s about starting a discussion on When-To-Branch and When-To-Work-On-The-Trunk (and When-To-Create-Tags).

Let me being by saying - It’s all about choices, opinions and which way works best your project. In the few years of using CVS (and then SVN) I’ve seen multiple ways in which the Trunk, Branches and Tags are used by multiple teams. Every team has its own “best practice” (the whole idea of a "best" practice is relative, because 'best' depends on so many factors :)) around when they should work on the trunk, when they tag their builds and when they create branches. Here are two most common approaches I’ve usually seen being used depending on the scenario and the project:

Approach 1 - Work on the Trunk, Tag on Each Release / Revision and Branch for Experiment

I’ve personally used this approach in projects where we have very frequent and agile releases. The approach basically involves:

  1. The team constantly working on the Trunk.
  2. The trunk is tagged at Every Revision as well as at Every Release so that a snap shot of any version can be obtained.
  3. The team continues to work on the trunk after the tagging. 
  4. If there’s anything the team wants to try out, they branch the code and merge back the changes to the trunk when done.

At any given day the Repository looks much like this:

  

What I love about this approach:

  1. The Simplicity and Speed at which team can work - Really helpful if you’re going to have weekly tags.
  2. Allows a “Branch-When-Needed approach” and leaves the branching-for-trying-out-complex-feature-implementations on Developers rather than imposing a strict policy on when to branch. They can freely do it as and when they want it.
  3. Easy to understand and work on, specially for VSS developers .
  4. Not having to change your Working Folder for each version change, since each developer is just concerned with the trunk and his own branches.

Cons:

  1. The Trunk may not always be functionally tested or 100% stable.
  2. A colleague brought this up during a discussion and his point was completely valid – “You can’t create patches for Release 1.1.1 and check it in to 1.1.1, if your trunk is at Release 1.2.” – (He’s all for Approach 2, I describe later in the post). Point valid and taken :) but I’m still not fully convinced that you really want to do something like that in a project that’s moving ahead really fast and throwing out revisions every week and a release every month or so; or for that matter, I'm not really sure if you want to do that for any project. 

Personal Opinion: I don’t like the whole Idea of applying SVN patches to a branch of a version that has already been released and then checking-in the patch to that branch. If an immediate patch needs to be applied to an already released version e.g. 1.1.1, I can still checkout the tag, work on the checked out tag, create a patch and then ship the patch. It should be possible to apply the patch on both, the primary trunk (which is now at 1.1.2) and the checked out version of tagged build 1.1.1 (which needs immediate fixing). However what I don’t agree to is the idea of applying the patch to a 1.1.1 branch and then checking in the changes to 1.1.1 specially after 1.1.1 has been officially released. It just creates more confusion and makes things product versioning more complicated. Personally, I feel this is exactly what service packs are for.

Approach 2: Always work on the Branch Approach - I’ve seen a LOT of development teams take this approach. People claim this is really helpful when you just want to do milestone based branches and tags. The Idea is:

  1. For Every Milestone / Release create a New Branch.
  2. Everyone works on that branch.
  3. No-one works directly on the trunk, and because no-one works on the trunk, the trunk always represents the previous stable release.
  4. When the current release is tested, changes are merged to the trunk.

At any given point the repository looks like:

What's good about this approach:

  1. The Peace of mind that my trunk is always tested and stable.
  2. You can create patches for an older version and check-in the changes to the old version by checking in to the correct branch, even after the product has been released. (Personally, I don’t think this is a Plus, but a lot of development teams use this approach and I've been told by more than one very capable developer that this approach has worked for them in multiple projects).
  3. As a colleague / mentor recently pointed out - If you are doing development of two versions in parallel this approach can help. I would have to agree on that one. We've done the "continue to fix the trunk while we work on a new version in a branch" approach and when I look at this approach from that perspective it suddenly starts making a lot of sense.
  4. Honestly, I can’t think of other Plus-Points for never-work-on-the-trunk approach.

Cons:

  1. The whole idea of never working on the trunk makes it complicated for developers to create their own branch especially when they are already working on a branch. E.g. I want to work on “Feature A” which I am not sure will work out. If I am already on a Branch for version 1.1.2, it makes it a little complicated for me to branch again and merge my sub-branch with the 1.1.2 branch before the 1.1.2 branch is merged with the trunk. (I hope I haven’t confused you already, the more sub-branches from branches that you created the more complicated things become :))

And then there are mix and match of the above two approaches that I’ve seen a lot of teams use. For example a team which creates tags when a build is released, even while following the Approach 2 I described above or the approach where you continue to fix the trunk which holds the first version while you work on a second version on a branch. I've also worked with a Team where every SVN ( / CVS) command was wrapped with a custom wrapper command using Unix shell scripts and use of clients like Tortoise wasn't permitted.

How you use the SVN Branches, Tags, the trunk and in fact, the source control system, is completely based on what works best for the project and the team. Feel free to Mix and Match approaches to control the chaos with predefined guidelines but make sure you leave room for flexibility and agility. 

There are no best practices! How to Manage code depends on the project, the team, how fast you want to go and multiple other scenarios. But it's always a good idea to have a set "SVN Working Practices" (notice the use of word 'working' instead of 'best' :)) for your Projects. Do you have a your SVN Working Practices Documented yet? If not, here's an excellent example you can use to get inspired. :)

Update [2/13/2007]: A colleague / mentor provided this link in a comment on my work blog. It's a practical repository of Branching Patterns for Parallel development. If you're interested in finding out about the various Branching Schemes out there and getting some guidelines on which scheme to use when, the article is a must-read!

posted on Friday, January 26, 2007 2:02:54 PM UTC by Rajiv Popat  #    Comments [0]