free html hit counter
Posted on: Saturday, March 3, 2007 by Rajiv Popat

Are You Stuck With a Problem? Divide and Conquer!

At a client’s office, almost a year ago, a very talented developer on their team thought it was interestingly funny when I said - “and that’s how you divide and rule”. “We call it Divide and conquer” – he laughed. [Being an Indian, at times my English tends to have a slightly different touch to it. :)]

“But you’re probably correct. You don’t just want to conquer. What you really want to do is to conquer and then control the situation and rule, for a long time.” - He added on.

No, we were not making a sinister plan to take over the world. We were just talking about how we can break one big Monolithic BizTalk Orchestration into smaller, easy to mange, easy to comprehend and faster orchestrations.

He had drawn up this Big Design of their existing approach on the left and then I had proceeded to draw smaller designs of multiple Orchestrations and how they would talk to each other on the right when I finally wrapped up the meeting with the line “divide and rule”. I was suggesting that he break his Orchestrations down to smaller moving parts so that when a part doesn't work - he can take it out - independently - and Analyze it!

I don't think any programmer disagrees with the fact that we should Code Smaller. A Lot of basic concept articles, teach a person who is starting out on his programming endeavors how he can Break Larger Pieces of Code into Smaller Pieces. But divide and conquer (/rule) design paradigm goes beyond just writing cohesive, small and easy to maintain functions and classes.

Every now and then I see even good developers forget the Divide and Conquer paradigm under pressure. Or they simply do not put it to practice for tasks other than coding. I've recently seen a very capable business analyst trying to learn build management by taking a CuriseControl configuration file and trying to configure CriuseControl to get the source code from SVN, fire the build, send email notification when the build fails and log the result in the formatted HTML report, all at once.

When I was called in to help, my first reaction was to wipe out all other code from the 50+ line config file - and leave just 5 lines that will have cruise control fire the build. Turns out, those five lines were broken. Once that was fixed we moved on to Getting the source code from SVN - another 3 lines which just worked. We kept fixing a part at a time and soon we were done in a couple of hours.

I've seen so many engineers (Highly capable ones. At times, I'm guilty of this too) run through more than 10 screens before they can reach a reported bug and just reproduce it. Then apply the fix, go through the screens all over again only to find out that the fix doesn't work. Then start thinking of a different fix. Over and Over again. Spending more than 5 minutes on every attempt, just to check if the fix worked. 

When we do this the time taken to fix a bug increases dramatically, the frustration level increases dramatically and the ability to think using a Pragmatic Approach comes down with every failed attempt. We start programming by co-incidence! In situations like these, Isolating the problem by replicating it using smaller code snippets and moving it out of your codebase so that you can analyze it separately, can help a lot. 

If you are trying to fix a bug that requires more than a couple of minutes to just reproduce and has taken more than 5 attempted fixes already, you might be better off isolating the bug by bringing it out of the codebase and reproducing it independently using simple code snippets. If you are finding a bug too complex to fix you're probably trying to fix more than one bugs at a time. If a feature you're building in your sprint seems too complicated you're probably trying to build two connected features simultaneously.

In the pressure of fixing a bug that refuses to get fixed or in the excitement of writing code for something fairly complex, it's very easy to tell yourself - "I'm almost there!" - and it's easy to spend away hours or sometime even days - by continuing to tell yourself that, not realizing that you're in fact, getting nowhere!

If something takes much longer to fix or build than you initially expected or you're starting to get frustrated with a problem - Divide and Conquer! And then, once you've conquered the problem, enjoy your Rule over the situation! :) 

posted on Saturday, March 3, 2007 7:44:47 PM UTC by Rajiv Popat  #    Comments [0]
Posted on: Tuesday, February 13, 2007 by Rajiv Popat

Reading XML Files with PowerShell (Monad)

A couple of months ago I had to make a difficult choice. A choice between Vista and Powershell (I still need time getting used to this name:)). Vista had officially RTM'ed and Powershell for Vista wasn't available. It was a difficult choice - wait on XP/2003 and continue to use Powershell or Move to Vista and wait for Powershell to become Vista Ready. I decided to move to Vista.

Recently, after about a couple of months, exactly as promised by Jeffrey Snover, the Moand team has Vista'ed the Powershell Installer and that means I can have both Vista and Powershell now. To add to that, I can totally continue to Skin Powershell and make it look beautiful, just like my Vista. And that's very important! Right? :)

I've been playing a lot with Powershell and now I think it's time for me, to put it to some real use, in project deployments on QA / Production.

One of the problems I always had while writing anything about PowerShell is that there are just too many good things to talk about in there and almost anything that could have been written about, has been written by someone else. That's one of the reasons (excuses? :)) why I haven't been writing a lot about it. For now, I'm just going to start by talking a little bit about XML and how you can read XML files in Powershell.

Let's just assume that we have the following simple XML file which lets me run various Uninstall and Install SQL scripts on the database depending on the Application version that I am pushing to the server.

<version name="1.0.0">
<script UnInstallScript="uninstall1-1.sql" InstallScript="install1-1.sql"/>
<script UnInstallScript="uninstall1-2.sql" InstallScript="install1-2.sql"/>
<version name="2.0.0">
<script UnInstallScript="uninstall2-1.sql" InstallScript="install2-1.sql"/>
<script UnInstallScript="uninstall2-2.sql" InstallScript="install2-2.sql"/>

Typically, creating a database project, a command file and then a custom Console application in C# that reads this XML file and fires corresponding SQL Scripts might work, but it takes some time. Let's use Powershell to see how easy it is to read this XML.

Let's start with simplest approach first. XML support is pretty neatly baked into Powershell. So assuming this file was called Mockfile.xml, I can just create a XmlDocument object using the following command at my Powershell window:

PS E:\> [xml] $MyXmlDocument = Get-Content MockFile.xml

That's it. The XMLDocument is now available in this instance variable. Let's take a look at what the variable $MyXmlDocuent contains:

PS E:\> Get-Variable MyXmlDocument

NameValue Value
---- -----
MyXmlDocument #document

PS E:\> $MyXmlDocument


We're all set to read from the document now. Let's try to get to various nodes on the XML File. Because, everything returned by Powershell is an object, to get the value of the Node all I Need to do is access the attribute within the object. For Example, to get all values of version nodes inside the the DBScripts Node and store it inside a separate variable I could just do:

PS E:\> $MyVersions = $MyXmlDocument.DBScripts
PS E:\> $MyVersions

{1.0.0, 2.0.0}

Notice how I access the DBScripts Node and assign all version nodes inside the root node to a different variable. To see value value of $MyVersions variable that I just created I just type it in and Powershell confirms that it now has nodes pertaining to both versions in this newly created variable.

Now let's assume I need to reach the version node with the Name 2.0.0. and access all the script files in there. To do this let's quickly take a look at the version node collection which was returned by the above command:

PS E:\> $MyVersions.version

name script
---- ------
1.0.0 {script, script}
2.0.0 {script, script}

Now to directly access the version node where the name is 2.0.0, let's use PowerShell filtering and let's pipe the output of version nodes to the the Where-Object commandlet:

PS E:\> $SecondVersion = $MyVersions.version | where-object {$ -match '2.0.0'}
PS E:\> $SecondVersion

name script
---- ------
2.0.0 {script, script}

PS E:\> $SecondVersion.script

UnInstallScript InstallScript
-------------- -------------
uninstall2-1.sql install2-1.sql
uninstall2-2.sql install2-2.sql

Once I've piped the output of all my version attribute to the where-object commandlet and assigned the final output to another variable I inspect the attribute the variable contains by typing $SecondVersion and then peek into it's script collection by doing $SecondVersion.script. If this was a deployment script, the above command would have given me all the scripts I need to run to install the version 2.0.0 of my application. I could easily convert this to a Powershell script where the version number comes in a parameter. The script can go ahead and figure out which SQL script it needs to run and then can run them. However, the focus of this Post is to concentrate of reading XML files, so let's try to do everything we just did using just two lines of code:

PS E:\> [xml] $MyXmlDocument = Get-Content MockFile.xml
PS E:\> ($MyXmlDocument.DBScripts.version | where-object {$ -match '2.0.0'}).script

UnInstallScript InstallScript
-------------- -------------
uninstall2-1.sql install2-1.sql
uninstall2-2.sql install2-2.sql

Sweet! Don't you think? :) Wait, there's more! You could actually do this using a single command at the command prompt if you wanted to:

PS E:\> (([xml] (Get-Content MockFile.xml)).DBScripts.version | Where-Object {$ -match '2.0.0'}).script

UnInstallScript Installscript
-------------- -------------
uninstall2-1.sql install2-1.sql
uninstall2-2.sql install2-2.sql

And then of course you could iterate through this list by using the ForEach-Object commandlet if you wanted to:

PS E:\> (([xml] (Get-Content MockFile.xml)).DBScripts.version | Where-Object {$ -match '2.0.0'}).script | ForEach-
Object { "Here I Could Run : " + $_.InstallScript }

Here I Could Run : install2-1.sql
Here I Could Run : install2-2.sql

PS E:\>

Or you could just access a particular element by it's zero based index:

PS E:\> (([xml] (Get-Content MockFile.xml)).DBScripts.version | Where-Object {$ -match '2.0.0'}).script[1]

UnInstallScript InstallScript
-------------- -------------
uninstall2-2.sql install2-2.sql

PS E:\>

And did I mention that two code snippets above are just one line commands? You could of-course turn this into a script depending on what you are trying to do. But what if you were not very comfortable with this approach and wanted to use familiar XPath to access XML nodes? Don't run away! You can do that too. This MSDN article shows you how.

Here's wishing you Happy-XML with Powershell!

posted on Tuesday, February 13, 2007 1:21:20 PM UTC by Rajiv Popat  #    Comments [0]
Posted on: Sunday, February 11, 2007 by Rajiv Popat

The Art of Throwing Away Code

Andrew Hunt and Venkat Subramaniam use the Turkish proverb, “No matter how far you’ve gone down the wrong road you’ve gone, turn back.”, rather elegantly in their book “Practices of an Agile Developer”. They comment:

"That Turkish proverb above is both simple and obvious—you’d think it would be a guiding force for software development. But all too often, developers (including your humble authors) continue down the wrong road in the misguided hope that it will be OK somehow. Maybe it’s close enough. Maybe this isn’t really as wrong a road as it feels. We might even get away with it now and then, if creating software were a linear, deterministic process—like the proverbial road. But it’s not."



Software development is about making choices. Starting from selecting the underlying frameworks your code will run on, tools you will use, a host of other choices and above all, the assumptions your code will make.

We make choices at every step. While we try our best to make our choices based on objective and pragmatic thinking, the fact is that our choices are based on the facts and requirements available at any given time. And of-course, in a real world, these change. Very often. Then, there is the pressure to ship and every developer needs to compromise some level of perfection for the sake of shipping!

Making just a couple of wrong choices can put you on the wrong way. You can try your best to avoid getting on the wrong way. I wish you luck. But the truth is, no matter how hard you try, you will take a couple of wrong turns once in a while. Building software is like driving on a confusing mesh of freeways, with an outdated map, that changes every five minutes, and having only a very general idea of where you want to get to. At some point or the other, chances are that you will take a couple of wrong exits and will get lost.

In a recent mail trail team members of a project and me discussed fixing 3 bugs in first few iterations of module. The module, which was a very small module of a large web based product, consisted of just 3 aspx pages. While these discussions were continuing, I received an email from a person who was assigned the responsibility of fixing these 3 bugs. He had completed his initial analysis on what it would take to fix those 3 bugs and this email contained his suggestion on how these bugs can be fixed. The email contained just a single line of text – “I think the code needs to be re-written.”

I consider emails of this sort particularly helpful. They tell me that we’re failing fast (And for those who didn’t click that link, that’s a good thing :)). Emails of this sort, tell me that even before we go to Testing, members in the team have actually figured out that we’ve just taken a wrong way and have raised the “wrong way” sign:

That just makes turning back very easy.

In this particular case, after a quick follow-up meeting on the mail-trail everyone agreed that we needed to fix a couple of bugs on a page, refactor a page ruthlessly and rewrite just one page because re-writing it would be faster than fixing it and would make things much better.

While spending hours and hours in trying to put band-aids on code that doesn’t work, or is badly designed, at rare occasions it’s a good idea to start fresh. In other words, “turn back”.

While turning back, works on micro-level modules, classes, small functions or code snippets, at a macro-level / project-level, more often than not, turning back is not a viable option.

At a product / project level, Joel Spolsky feels this is one of those “Things you should never do”. He explains:

"We're programmers. Programmers are, in their hearts, architects, and the first thing they want to do when they get to a site is to bulldoze the place flat and build something grand. We're not excited by incremental renovation: tinkering, improving, planting flower beds.

There's a subtle reason that programmers always want to throw away the code and start over. The reason is that they think the old code is a mess. And here is the interesting observation: they are probably wrong. The reason that they think the old code is a mess is because of a cardinal, fundamental law of programming:

It's harder to read code than to write it"

He goes ahead to explain why taking a huge project and attempting to rewrite it from scratch is not such a great idea:

"The idea that new code is better than old is patently absurd. Old code has been used. It has been tested. Lots of bugs have been found, and they've been fixed. There's nothing wrong with it. It doesn't acquire bugs just by sitting around on your hard drive."

But then, how do you turn back, especially if you have made too many bad choices and your project is a huge one? A Wise comment on this forum explains how: 

"Get together a refactoring plan. This shows how you can go from a mess to something useable in incremental steps.

I've never seen a project where absolutely every single line of code was a waste. Even if the architecture underneath is flawed, the UI might be okay. So keep the UI and refactor what goes on underneath.

It's always going to be quicker than a rewrite."

Pick and choose what you need to throw away, when you need to throw it away and how much of it do you really need to throw away. While you do that, don’t forget to keep the risks-of-throwing-away-code to a minimum. When you can throw away small snippets and fix things by rigorous yet phased Refactoring, don’t attempt a monolithic rewrite by throwing away all your code.

Where am I trying to get with this? If you haven’t already noticed, I’m just trying to refactor the proverb I started off with. So, here’s my refactored version: 

“No Matter how far down the wrong road you’ve gone, find the shortest, safest and smartest way to the right road.” - If writing code is an art, throwing it away is an art too! 

posted on Sunday, February 11, 2007 7:12:45 PM UTC by Rajiv Popat  #    Comments [0]
Posted on: Tuesday, February 6, 2007 by Rajiv Popat

Fixing Package Load Failure Error with Windows Workflow Foundation and Visual Studio

I've often said that the really good part about Windows is What can be done, or sometimes... what happens ("seemingly automatically"), can be undone (and explained). The same is also true for Visual Studio 2005. At rare occasions there are errors in the IDE which seem magical and mysterious. With the right debugging techniques, commands, some time the mysterious errors can be fixed and explained.

I've been busily coding away at Windows Workflow Foundation for a few months now. Currently I'm using the final released version of Windows workflow foundation with the released version of Workflow extensions for Visual Studio, and my experience with WF has been very good. So when this strange error popped up today without any particular reason, I was a little surprised.

The error wasn't very helpful in describing what was going wrong. All it stated was: Package Load Failure. It basically said - "Package 'Microsoft.Workflow.VSDesginer.DesignerPackage, Microsoft.Workflow.VSDesigner, version=, Culture=neutral, PublicKeyToken=31bf3856ad364e35' has failed to Load Properly (GUID = {some_guid}..." and then it went on to ask me If I wanted to disable my workflow foundation extensions for now.

Even though my workflow project opened and worked perfectly this was just an irritating error which popped up every time I started Visual Studio 2005. Determined to put an end to this harmless yet irritating error, I started my investigation and Google searches.

I started off with basic commands like - "devenv.exe /setup" - which did not fix the issue at hand. Then there were posts out there which suggested that I delete my native images and regenerate them using the ngen.exe tool. That solution did not seem to work either.

After a few hours of Google searches I came across this post which basically described what the issue was. Turns out, a few days ago I had tried my hand at writing Visual Studio Add-ins and had forgotten to delete an Add-In I had written. Remains of my HelloWorld Addin existed in "C:\Users\admin\Documents\Visual Studio 2005\Addins" on my Vista Partition.

All I had to do was to delete the ".Addin" file (which btw, had nothing to do with Windows Workflow Foundation, so it's a little difficult to suspect this would be the cause of the problem) and as soon as I had done that Visual Studio Started without any Windows Workflow Extension Add-in errors.

If this doesn't do the trick for you - here are few other commands / resources which are usually helpful in catching and fixing Visual Studio.NET 2005 IDE issues:

  1. devenv.exe /setup - very useful for a host of other IDE setting related problems or if you just want to restore VS settings to their original state. Did not help in my case because the failed add-in was something I had written myself and my problem had nothing to do with visual studio settings. In my case, the settings were just fine, even though I had initially suspected that they had gone corrupt. 
  2. ngen.exe - Aaron Stebner's Post describes this pretty elaborately. The basic idea here is to delete your Visual Studio Native Image files and regenerate them using this tool.
  3. devenv.exe /log - this generates XML log which can be quite helpful in trouble-shooting various startup issues with the IDE.

And if you're still struggling Aaron Stebner also provides a Definitive list of workarounds for Package Load Failure errors in Visual Studio 2005, here.

We spend so much of our time with the IDE everyday writing code. Mysterious errors like give us a chance to look under the hood and see what's going on. It's just like the joy of fixing that little problem in your car, yourself. After all knowing how to fix minor problems with your car, or knowing how to replace a flat tire, is also important besides knowing how to drive. Isn't it? :)

posted on Tuesday, February 6, 2007 6:02:34 PM UTC by Rajiv Popat  #    Comments [0]
Posted on: Monday, February 5, 2007 by Rajiv Popat

Why Does Fred Program By Coincidence

Andy Hunt and David Thomas in their book “The Pragmatic Programmer” make an interesting observation about a persona Fred:

"Suppose Fred is given a programming assignment. Fred types in some code, tries it, and it seems to work. Fred types in some more code, tries it, and it still seems to work. After several weeks of coding this way, the program suddenly stops working, and after hours of trying to fix it, he still doesn't know why. Fred may well spend a significant amount of time chasing this piece of code around without ever being able to fix it. No matter what he does, it just doesn't ever seem to work right.

Fred doesn't know why the code is failing because he didn't know why it worked in the first place. It seemed to work, given the limited "testing'' that Fred did, but that was just a coincidence. Buoyed by false confidence, Fred charged ahead into oblivion. Now, most intelligent people may know someone like Fred, but we know better. We don't rely on coincidences---do we?

Sometimes we might. Sometimes it can be pretty easy to confuse a happy coincidence with a purposeful plan"

During a discussion with one of my early project Managers, a Mentor and a person I really look up to, he coined Fred’s symptoms and gave it an appropriate name – he calls this the “Junior Developer Syndrome” and feels that every programmer was once Fred.

He also describes another very important feature of Fred. According to him (I couldn’t agree more), during these “hours of trying to fix it”, this Fred guy usually doesn’t tell anyone there’s a problem.

Like all other programmers the first couple of years of my coding life went in getting rid of the Fred within me. These years also taught me lessons all of us have learnt at point in our life. This post is an attempt to organize my own thoughts and try and understand why Fred programs by co-incidence.

There are lots of other complex factors that go into software development life; however, to keep this post simple let’s consider that Fred is worried about the tree most evident factors in a software development project.

  1. Quality of Code.
  2. Time Needed to write Code.
  3. Pressure to ship (from Stake-holders).

I’ll just go ahead and use graphs to illustrate how these three typically play together.

The Graph Plots Time needed to code against the quality of code. Spending a little bit of more time in understanding the Technology and figuring out the “why the code works” part gives a tremendous boost to the quality of code by investing just a little bit of time. The initial tweaks and optimizations are easily achieved by spending a little bit of time on the technology and understanding why the code works and how it works. The Graph depicts this as the distance between points A and B. i.e. [AB].

The most typical example of this I’ve seen is ASP.NET caching. For example, I’ve been on projects where a 4 second Page load was brought down to 300 milliseconds by just spending a day on Caching. In other words spending [AB] time gave us a [XY] increase in the quality of code.

After you’ve spent that initial time however, the law of Diminishing returns starts to kick in and every other tweak on code takes more and more time. In the same project I mentioned earlier we spent over a week of performance tuning to bring the 300 milliseconds down to less than 150. In other words, we ended up spending [BC] time to increase the quality of code by [YZ].

Now the obvious question is if by spending just [AB] worth of time learning about the “why” and the “how things works under the hood” of the technology on which Fred is working on, if Fred can get and [XY] gain in the quality of code why doesn’t he invest that time in making his code better?

In this past couple of years I've discussed this (in one way or the other) to multiple Senior Associates, Technical Architects and Project Managers. Some have told me that Fred is just stupid. Others claim that Fred just works for a Pay-check and [OX] worth of quality is usually good enough to get the project go through client acceptance. Fred couldn’t care less about code.

Others however, mentioned a reason that sounds more likely – The third factor I talked about earlier in the post - Pressure to ship – that is what causes Fred “Program by Co-incidence”.

Notice the graph above. Here the pressure to ship show up suddenly as your product / project starts getting shape or starts generating excitement. As you move from point E to F tweaking your code, the pressure to ship slowly growing up by [PQ]. As you move from F to G and the Shipping-Date / Dead-Line / Committed Date comes closer the pressure to deliver shoots hugely by [QR].

Now let’s merge both of these graphs.

Merging these two graphs answers some of the questions and gives a new perspective into – Why does Fred write Bad code. Think about it a little more and it makes so much more sense.

The problem with Fred is that he considers his code ready to ship as soon as he senses some pressure to Ship from Stake-holders. He considers his code ready-to-ship at point X. Fred wants to see the Stake-holders happy. He’s not a bad developer! Or, is he?

The other extreme are people who will not ship till point Z. People who will spend weeks to fine-tune a tiny performance issue that is causing a round-trip to take 15 milliseconds more than what it should take. This kind of a developer runs the risk of Never Shipping, because the Green curve never becomes parallel to the X-axis. In other words, no matter how much you tune and tweak your code there is always room for improvement.

So when should you ship?

During one of my first projects in my development life (when I was very young and enthusiastic :)) I shipped a project at point X and paid the price for it. The code I shipped very soon became my problem; within a couple of months I saw myself taking support calls and fixing bugs by working over 18 hours a day; hating the code I had written in a hurry and fixing it where possible. It could have bitten me really bad, but thank god it didn’t. We fixed all major issues that were reported and No-one was killed. There was nothing about that project I enjoyed in particular, but it taught me to strike a balance. It taught me to ship at Point M because M is an idle point to ship.

Shipping at point M in the above graph, involves it's own set of challenges though. It means maintaining constant communication with the Stake-holders, reporting accurate status, remaining agile and selling the idea that Quality would mean lesser cost in the long run to the Stake-holders even at the time when pressure to ship is constantly increasing.

What does this boil down to? While the book suggests some excellent code-related-techniques of how Fred can become a programmer who does not program by coincidence, in my opinion however, It also boils down the fact that to ship at M Fred needs to learn a whole host of other skills which have nothing to do with coding. Skills which Fred always considered unimportant as a developer. A mentor remarks - "Software development is not all about For loops. A lot of software development is about Managing Risks and so many other things". These are In fact, words of wisdom which deserve a different post for themselves.

posted on Monday, February 5, 2007 12:05:43 PM UTC by Rajiv Popat  #    Comments [1]
Posted on: Friday, January 26, 2007 by Rajiv Popat

The When and Why of SVN Branches, Tags and Trunk

At work we use SVN for all our .NET Projects. Being a Microsoft Developer I’ve moved between multiple source control systems during my development life. Starting with Visual Source Safe, I moved to CVS, then to TFS (during its Beta stages) and finally fell in love and settled down with SVN. The love was strong enough to convert all .NET developers I work with, over to SVN! :)

Even today I highly encourage all Microsoft developers I meet outside the workplace to try-out SVN. With Tortoise and Ankh, the whole Visual Studio Integration and Explorer Integration is very easy to get and it (almost) feels as easy as VSS and as powerful as CVS. But this post is not about plugging-in SVN and related tools. It’s about starting a discussion on When-To-Branch and When-To-Work-On-The-Trunk (and When-To-Create-Tags).

Let me being by saying - It’s all about choices, opinions and which way works best your project. In the few years of using CVS (and then SVN) I’ve seen multiple ways in which the Trunk, Branches and Tags are used by multiple teams. Every team has its own “best practice” (the whole idea of a "best" practice is relative, because 'best' depends on so many factors :)) around when they should work on the trunk, when they tag their builds and when they create branches. Here are two most common approaches I’ve usually seen being used depending on the scenario and the project:

Approach 1 - Work on the Trunk, Tag on Each Release / Revision and Branch for Experiment

I’ve personally used this approach in projects where we have very frequent and agile releases. The approach basically involves:

  1. The team constantly working on the Trunk.
  2. The trunk is tagged at Every Revision as well as at Every Release so that a snap shot of any version can be obtained.
  3. The team continues to work on the trunk after the tagging. 
  4. If there’s anything the team wants to try out, they branch the code and merge back the changes to the trunk when done.

At any given day the Repository looks much like this:


What I love about this approach:

  1. The Simplicity and Speed at which team can work - Really helpful if you’re going to have weekly tags.
  2. Allows a “Branch-When-Needed approach” and leaves the branching-for-trying-out-complex-feature-implementations on Developers rather than imposing a strict policy on when to branch. They can freely do it as and when they want it.
  3. Easy to understand and work on, specially for VSS developers .
  4. Not having to change your Working Folder for each version change, since each developer is just concerned with the trunk and his own branches.


  1. The Trunk may not always be functionally tested or 100% stable.
  2. A colleague brought this up during a discussion and his point was completely valid – “You can’t create patches for Release 1.1.1 and check it in to 1.1.1, if your trunk is at Release 1.2.” – (He’s all for Approach 2, I describe later in the post). Point valid and taken :) but I’m still not fully convinced that you really want to do something like that in a project that’s moving ahead really fast and throwing out revisions every week and a release every month or so; or for that matter, I'm not really sure if you want to do that for any project. 

Personal Opinion: I don’t like the whole Idea of applying SVN patches to a branch of a version that has already been released and then checking-in the patch to that branch. If an immediate patch needs to be applied to an already released version e.g. 1.1.1, I can still checkout the tag, work on the checked out tag, create a patch and then ship the patch. It should be possible to apply the patch on both, the primary trunk (which is now at 1.1.2) and the checked out version of tagged build 1.1.1 (which needs immediate fixing). However what I don’t agree to is the idea of applying the patch to a 1.1.1 branch and then checking in the changes to 1.1.1 specially after 1.1.1 has been officially released. It just creates more confusion and makes things product versioning more complicated. Personally, I feel this is exactly what service packs are for.

Approach 2: Always work on the Branch Approach - I’ve seen a LOT of development teams take this approach. People claim this is really helpful when you just want to do milestone based branches and tags. The Idea is:

  1. For Every Milestone / Release create a New Branch.
  2. Everyone works on that branch.
  3. No-one works directly on the trunk, and because no-one works on the trunk, the trunk always represents the previous stable release.
  4. When the current release is tested, changes are merged to the trunk.

At any given point the repository looks like:

What's good about this approach:

  1. The Peace of mind that my trunk is always tested and stable.
  2. You can create patches for an older version and check-in the changes to the old version by checking in to the correct branch, even after the product has been released. (Personally, I don’t think this is a Plus, but a lot of development teams use this approach and I've been told by more than one very capable developer that this approach has worked for them in multiple projects).
  3. As a colleague / mentor recently pointed out - If you are doing development of two versions in parallel this approach can help. I would have to agree on that one. We've done the "continue to fix the trunk while we work on a new version in a branch" approach and when I look at this approach from that perspective it suddenly starts making a lot of sense.
  4. Honestly, I can’t think of other Plus-Points for never-work-on-the-trunk approach.


  1. The whole idea of never working on the trunk makes it complicated for developers to create their own branch especially when they are already working on a branch. E.g. I want to work on “Feature A” which I am not sure will work out. If I am already on a Branch for version 1.1.2, it makes it a little complicated for me to branch again and merge my sub-branch with the 1.1.2 branch before the 1.1.2 branch is merged with the trunk. (I hope I haven’t confused you already, the more sub-branches from branches that you created the more complicated things become :))

And then there are mix and match of the above two approaches that I’ve seen a lot of teams use. For example a team which creates tags when a build is released, even while following the Approach 2 I described above or the approach where you continue to fix the trunk which holds the first version while you work on a second version on a branch. I've also worked with a Team where every SVN ( / CVS) command was wrapped with a custom wrapper command using Unix shell scripts and use of clients like Tortoise wasn't permitted.

How you use the SVN Branches, Tags, the trunk and in fact, the source control system, is completely based on what works best for the project and the team. Feel free to Mix and Match approaches to control the chaos with predefined guidelines but make sure you leave room for flexibility and agility. 

There are no best practices! How to Manage code depends on the project, the team, how fast you want to go and multiple other scenarios. But it's always a good idea to have a set "SVN Working Practices" (notice the use of word 'working' instead of 'best' :)) for your Projects. Do you have a your SVN Working Practices Documented yet? If not, here's an excellent example you can use to get inspired. :)

Update [2/13/2007]: A colleague / mentor provided this link in a comment on my work blog. It's a practical repository of Branching Patterns for Parallel development. If you're interested in finding out about the various Branching Schemes out there and getting some guidelines on which scheme to use when, the article is a must-read!

posted on Friday, January 26, 2007 2:02:54 PM UTC by Rajiv Popat  #    Comments [0]
Posted on: Saturday, January 20, 2007 by Rajiv Popat

Protecting Intellectual Property Vs. Trusting Employees

If you run a company that owns a product, or have put your mind and soul into writing a product that you believe will make you or your organization big bucks and success in the long run, you’re probably worried about protecting your intellectual property.

Lately, I’ve been hearing a Lot about Intellectual Property and what Organizations should do to protect their intellectual property. Multiple discussions have taken place, with multiple people from different Organizations. Just to give an idea of the crowd I’ve been discussing this with, let me introduce an  acquaintance who works in a 400+ person, Indian solution provider as a developer.

Another individual is a Lawyer in a 100+ employee Organization having offices in US and multiple offshore countries including UK and Europe. Just to bring a slightly different perspective to discussion I’ve also included comments from a very old client that I worked with – who had a no-work-from-home policy. And then there are guys from IT / Administration world back from MCSE days, that I am still in touch with and who work with some really medium-to-small-sized development shops.

Before I go any further however, let me take of my ‘I-know-everything-about-this’ hat off and hide it somewhere. Because, this is my blog which runs on a server I pay for, it goes without saying that I will present my opinions. However, this is also a good time to bring to everyone’s notice that my opinions (by their very nature) here are highly opinionated and may be as far from correct as anything can be. But after hearing so much about this topic I really feel that I have to post about this. If nothing else, this post is an attempt to take a look at one problem from multiple perspectives, including mine. :)

"They are not allowed to carry USB drives to work. No Floppies, no CDs, they code on desktops with 512 Megs of RAM, unless of course, someone can justify that he needs a Gig of RAM for his project. 90% of Internet is blocked from them including Yahoo mail, Hotmail, Messengers or anything that can potentially be used as file transfers."

An acquaintance back from my IT Days describes. He works at an Indian IT consultancy firms which is small enough for every employee to know every other employee on a first name basis. By “They” he is referring to the developers. There’s a particular advantage of having worked in various departments in my early part of professional life. I associate with both IT and Development folks equally well, and completely “get-it” when the folks in one camp refer to the folks in other came as “They” :)

"In fact, our IT takes a Pessimistic approach to security. They start by blocking everything. Employees over years have requested specific site to be unblocked which has resulted in a fairly large database of safe sites which have been unblocked. Once a request is filed, it is analyzed to see if the site offers any mechanisms to transmit confidential data. Analysis is also done on why this site should be opened up. There are cases, where we’ve asked for a specific site to be opened up because we wanted to read an article and have received an email attachment in reply with the content on the article attached and our request to open-up the site denied, mainly because the site provided free mailing services. The concern is that we would email code to ourselves."

These are measures a 400+ employee consultancy firm takes to protect its Intellectual Property. This comes from a Developer and here “they” of course refers to the IT folks :). The third perspective is slightly Non-Technical coming from a lawyer who works for a relatively smaller development firm. 

"The key to this is Making employees sign Non-Disclosure-Agreements (NDA’s), Non-Compete agreements and copyrighting your code. Once these measures are in place, the real work needs to begin - the technical departments, like the IT needs to move in and enforce measures that code-theft cannot happen even if an employee wants to commit a theft e.g. disabling their USB drives, not giving them CD/DVD-Writers etc. Making the employees sign is the easy part. Setting up systems so that you don’t have to be at the mercy of mutual trust with your employees is the difficult part"

Another person at fairly small US based organization, sites an example of an employee running away with a laptop and some code and the team being worried for a couple of weeks till the employee was tracked down and they had confirmed that he hadn't released the source code to anyone.

After I listening to these discussions, some-thing deep down somewhere kept telling me that there’s something wrong, somewhere. As if one side of the story is being ignored. One thing that seems to resonate through all these comments and remarks is – "It's difficult to trust your Employees. Doing that will always mean big trouble". In an attempt to discover the other side I decided to cling on to Google and go on a search for other opinions.

My Ideas on this topic are very different from the ones that had been brought to the plate so far, and Google seemed like an excellent tool to figure out if there are others who had similar thoughts and to figure out if my thoughts are working out for them too.

The first result is an interesting instance of stealth and sleuthing – what’s most interesting about this article is that it sticks to the side of story that’s been presented by all other quotes in this post so far, but ends with lines that come very close to the other-side of the story, therefore striking a really nice balance. The article includes an interesting and (in my opinion) a very true remark:

"It is impossible to provide for a completely foolproof system... To devise a foolproof system, you would need a set of people working on it...This set could have a thief in its midst too. In the ultimate analysis, everything works on trust. After all, software employees are capable of anything."

Why I particularly like this remark is because it addresses the problem from a real perspective and considers the fact, that when dealing with developers companies are dealing with smart individuals who are capable of writing highly secured and scalable systems. It goes without saying that any developer worth his salt, who is capable of writing these systems, is also capable to stealing code or in fact, anything, if he really wanted to do it, specially if he's a little lucky. There is no such thing as a fully secured process or system! And that’s one thing we tend to forget when talking about Systems that we claim will replace (or enforce) human trust and dignity through security.

There are tons of articles out there which tell you that you cannot trust your employees, not even your administrators – but ones which tell you that it’s OK to place a little bit of trust in sensible programmers who you hired in the first place, are few and difficult to find!

After spending some time on Google searches I finally landed on some sound-advice that came close the kind of answer I was really looking for. A very wise comment on this forum states:

"It seems you've learned firsthand that you can't *really* protect your IP. That said, you could talk to a lawyer and get some more specific advice. Certainly requiring your hires to sign a non-disclosure agreement is nothing onerous. But I think the best thing to do is hire people with integrity and all. Be so successful that nobody in their right mind would think about splitting, and that none of your customers would think about switching"

These are not my words, but if I was to say something on this topic this is exactly what I would say and then I would add a few more numbered points (read on).

  1. You cannot prevent your employees stealing your code unless you’re planning on frisking people: In each of the remarks people made in their discussions with me, and as they were describing their security-measures to protect their Intellectual Property, I could instantly think of more than one ways by which a developer could easily steal code if he really wanted to. The comment in this forum, that you cannot prevent employees from stealing code unless you’re planning on frisking people, seems quite true and at-least I agree to it.
  2. What is your Intellectual Property, by the way? As a developer I spend countless hours, throwing away code that I’ve written in the past and writing better code. Those of us, who believe in TDD, refactor ruthlessly and make that a way of life. A lot of the projects I’ve been involved with are re-writes of existing systems. So, as developers, if throw away so much code, is code what you really want to protect? Is it even wise to consider just your code your real (and only) form of intellectual property?

    I for once want to, believe that an Organization's Real Intellectual Property is the knowledge that was gathered in writing that code and the individuals who retain that knowledge. That’s what I would be really interested in protecting.

    In a discussion one of my Managers once asked me how long my team would take if I was to scrap every single line of code I’ve written for a product and start fresh. My reply was that it would take half the time and I would come up with a product that would be faster and much more feature rich.

    As developers, we grow with each project, module and problem that we solve. I personally feel that it’s this growth that organizations and individuals should be striving to protect. Not just Code!
  3. A Lot of code isn’t complete or looses meaning out of context: some years ago; a part of Windows NT code base and a part of Windows 2000 code base was leaked out. Everyone in the community talked about it. People discussed it. Some of us even downloaded it and took a peek at it. People analyzed it, more out of curiosity than anything anything else. But that was it. People couldn’t build the code. Of-course they couldn’t compile Windows NT / Windows 2000 out of it. The code was incomplete and it was out of context.

    There was a particular client that I worked with at an early part of my life, who wouldn’t let any consultants work from home because the DBA was particularly concerned about a Table in the new system which sucked legacy data and had an Item-Id column containing millions of items Ids. While, I completely understood and respected the DBA’s passion to protect data from leaking out, the fact remains that 2 million random numbers (which happen to be a list of item ID) are utterly useless for me, because they are out of context. There is nothing I could have done with a huge list of their Item IDs even if I wanted to. It took quite a bit of convincing to get the permission to be  able to work from home so that I could fix bugs I was really concerned about during late nights and weekends and in the end it worked out really well. We delivered a couple of weeks before the planned date and there was no loss of any intellectual property. 
  4. Shouldn’t you focus on what’s coming in rather than focusing what’s going out? In all the arguments for strict systems to protect intellectual property the focus seems to be on “what is going out of the organization” - either an employee or code. Why not spend more time on focusing on employees that are coming in the organization and create processes for hiring people with Dignity and Maturity. Something makes me feel that if some of these Organizations, that spend huge amounts of time in building these extreme measures, spent as much time in thinking about the quality of employees they're hiring, they wouldn't have to worry about Protecting Intellectual Property.
  5. I respect NDA’s and honestly, I don't mind signing them: I don't think any developer does. As one of the quotes above mentions "Making the employees sign is the easy part". It really is. And Important too.  Thought I should clarify that before I start getting emails from people I know and don’t know telling me that I am a moron who doesn’t care about protecting intellectual property and respecting NDAs. I completely understand a client's concern to protect their Data and Intellectual Property. I also understand policies of not letting people carry parts of projects or Data on laptops in some cases or projects where the sensitivity is high.

    However,  I would find it a little awkward, if I ever had to work in an organization that implementing policies and had systems which constantly keep telling me every-day, that I am a potential threat to the business. In my personal opinion, some of the comments I heard during these conversations sounded a little extreme which is what got me thinking.

I find the measures, mentioned by some of my friends, in this post, a little extreme. It's probably because of the fact that I've changed very few organizations in my professional life and have been lucky to work at Organizations (including clients and project-teams) which have very difficult interview processes but provide tremendous amount trust, freedom and liberty in the hands of their employees once they are a part of the team.

A sarcastic answer to “extreme measures and systems to protect your intellectual property” posted at this forum seems like a good way to end this post. A person with a sense of humor comments –

"I want to work for you people! A boss who considers me a significant threat to his business and acts as if I'm a thief waiting for the opportunity to strike would make me feel like such an appreciated member of the team."

Do you feel that you work at a place that considers you a significant threat to their business? Wondering what you can do about it? If you answered "Yes" to these questions, remember - you can either change your company, or you can change your company. :)

posted on Saturday, January 20, 2007 7:53:30 PM UTC by Rajiv Popat  #    Comments [2]
Posted on: Monday, December 25, 2006 by Rajiv Popat

Have You Made Your New Year's Resolutions for 2007 Yet?

Here is wishing anyone who reads this a very Merry Christmas, Happy Holidays and a very Happy New Year! It’s that time of the year again! Time for celebrations, rest, holidays, fun and above all, a time to get inspired!

A whole new year is near, and that itself, is a good reason to take a pause, look back and spend some relaxing time to plan for the year ahead. Personally, these times feel so very similar to those during my long 10+ hour flights, when the plane is standing silently on the runway and we’re all waiting for the takeoff; The plane staff are checking the pre-flight checklists, assessing the weather conditions, checking the flight-plans and we’re all ready to fly and get to where-ever-it-is we want to get.

Of-course, having a flight-plan for life is a very difficult. But, this is the time when most of us make New Year’s Resolutions before we take off for the year. Wikipedia states:

"The new year resolution is one example of the rolling forecast-method of planning. According to this method, plans are established at regular short or medium-term time intervals, when only a rough long-term plan exists."

Looking back at the History of New Year Resolutions, it was the Babylonians who started this whole concept. They believed that what a person does on the first day of the year will have an effect on that individual, throughout the year. The most popular Babylonian resolution was to return something they borrowed from a friend (usually, farm equipment).

Today, lists like the List of Top 10 New Year Resolutions seems to cover and claims to provide help with most of the resolutions that a lot of us tend to make these days. Then there are sites out there that also provide help in the form of a paid-service which is supposed to help you meet your New Years Resolutions. Are they effective? Honestly, I haven't tried any and hopefully, don't intend on trying them anytime in the future. 

Personally, I don’t find the whole idea of New Year's Resolutions very appealing. Most of the resolutions we tend to make during this time are way too optimistic and un-real. Maybe, that’s why they fizz out within a couple of months. I’m more inclined towards introducing agility into my life using Professional and Personal TODO Lists for Life and striking off items from these lists whenever life permits. I’m also into doing small and agile sprints of myself. I call this the “Your-Next-Version” approach.

The idea of versioning myself and then working for Self-Version started out during a great vacation this year when I spent a lot of time thinking about my plans of self improvements in the next 6 months. I wrote a lot about the things that I wanted to change and made a lot of so-called-resolutions. Then I announced in this blog that I was done doing an elaboration of “Rajiv 0.2” and construction would being soon and that people would notice the difference.

I’ve always said that too much Analysis doesn't really help and after posting about the problems with analyzing too much, I soon realized that I had ended up doing the same thing with my own versions. My idea of versioning myself and keeping constant track of the self-versions, could work, but if it was to work, it needed to be light-weight and not too-optimistic-big-plan-up-front kind of an approach. The approach was quickly tweaked to support agility.

Each version of “Me” would include just one or two new features. The features included in each version would be both simple and incremental - As an example, a feature could say:  

"Just because you can, don’t code for others. You’re not really helping them. Let them find their own answers – even if you know them already. Be a mentor, not a human search-engine!"

A version, typically consisting of not more than a couple of features, can take anything between a month to a couple of months to roll out and my life would continue as normal, except of course I would be really careful and serious about committing to and making just one or two features a part of my life. Once this was a habit, I would call this version complete. For the next version, I would move to next one or two features, which may be an extension of the features in current version that was just completed or something completely disconnected. This approach seems to be pretty effective till now.

This New Year, I suggest the Your-Next-Version approach as an alternate option to New Year’s resolution. "What’s the difference?" - You might be thinking. The difference is exactly the difference between Waterfall and Agile. A New Year resolution is too generic, long and waterfall-like. E.g. for my friends and colleagues who smoke, A typical New Year's Resolution is often something as generic as “Quit smoking” – (way too optimistic and most likely to fizz out within the first couple of weeks or maybe a couple of months).

With the Your-Next-Version approach, we tweak this slightly and just say - My Next Version / Sprint smokes a cigarette less than my current version. Whenever this is achieved and is a comfortable part of your life, you move to the next version / sprint where the features could be an extension of existing features (a couple of cigarettes lesser) or something completely disconnected. We’ve worked with quick sprints, releases and versions in the software world; isn't it natural to extend the same knowledge and use it to tweak ourselves?

Do you version yourself too? If not, doesn't a whole New Year seem like a good time to try this out? Keep your own versions small and keep them agile. Here’s wishing everyone a very merry Christmas, great Holidays and a beautiful New Year full of lessons, love, family-time, work, excitement, growth and lots of new, exciting and stable versions of your current self! :)

posted on Monday, December 25, 2006 5:17:58 PM UTC by Rajiv Popat  #    Comments [0]