10 interesting lessons from 1 interesting project

I’ve spent six interesting months on my current project and my association with it is at last coming to a close. The design phase is largely over and I can finally move on to other things.

Like some sort of technological Santa Clause, I skip from chimney to chimney spreading good cheer, pixie dust and sooty footprints as I go. I like to think I leave happy families in my wake, opening the little ‘presents’ I’ve left behind to shrieks of excitement (or is that terror? I never can tell). I do know they’ll be looking forward to seeing me once more, no matter how much coal they ended up with in their stocking this year :-).

Anyway, this current assignment has been a most interesting project; as in “May you live in interesting times” and “Why yes, Nurse, this is quite an interesting case”.

Yup, that kind of interesting.

It’s a pity that most of the lessons that this project have taught me are of the political, managerial and interpersonal variety than those of the technical type! I’ve noted some of them down in this post, more as a reminder to myself than anything else.

A quick overview


The project aims to develop a Java based data entry application for a large Swiss financial services firm. The onshore team was from a German company (part of the corporate group) and the offshore development was handled in Mumbai. There are around 20 devs, 5 designers and various testers, managers and other assorted mischief makers. The project is now over budget, delayed and the quality of the deliverable is not up to snuff. The only reason there’s any hope of completing in time is because individually, the team members are very highly skilled. It’s just that getting people to work together seems to be very difficult.

I joined the project just six months ago, when the good ship INS T******s had already struck ice and was beginning to list rather alarmingly. I came onto the scene just in time to miss the preceding bit where most of the mistakes had been made, but just in time to start seeing their effects start to become apparent. It’s no fun joining a project and immediately realising that you’re going to have to fight your way to the lifeboats before long! :-)

Mind you, it’s not been as bad as some projects I’ve worked on, but in many ways it’s been even more frustrating, because success was so close, and now it’s oh so far.


The 10 lessons

1. People have to be aligned to project success

If people can succeed at the expense of the project, then the project will suffer. If you make it almost necessary for them to bury the project to further themselves, then stand clear of the stampede for the shovels!

People are not intrinsically evil. They want to do the right thing and they will too, if given the chance. However, if you inadvertently stack the environment such that the only way they can advance their own position is by stepping on someone else’s face, then there will be a lot of people wandering around with bloodied mugs. It’s the responsibility of the project manager(s) to align individuals and teams properly, so everyone is pulling together towards project completion and success.

Fail to do so and you doom the project to delay and possible failure.

Some things to keep in mind:

1. There should be no gain in shunting blame between onshore and offshore.
2. Onshore should not be allowed to ‘succeed’ by leaving offshore behind and vice-versa.
3. Onshore should gain nothing by show-casing itself independently before the client and neither should offshore.
4. Onshore should be reassured that their roles and jobs are secure and that they will not be ‘Bangalored’.
5. Offshore should not attempt to aggressively infiltrate and displace onshore. See point 4.

2. Architectural Vision is vital

You don’t have to nail down each and every detail before you start the build. You can back-track once the build has started and update the design to reflect new learning’s. However, you cannot start the build without having a very clear picture of what you want, how you want it and how you’re going to get it.

The first thing I noticed and flagged when I joined the project was a lack of project vision. No one really knew how exactly things were going to interact with each other or how changes to one component were going to impact another. There were a lot of things which were just up in the air and it was anyone’s guess as to where they were going to land.

A major reason behind this was the lack of a proper POC (proof of concept). Had one been constructed, the entire design and build stages would have been several orders of magnitude easier. If there’s one thing I’ve taken away from this project, it’s this:

Don’t skip the POC.

3. Never split tasks and teams across geographies

Take a stale, grilled cheese sandwich and start pulling it apart. You’ll notice that there’re all kinds of tendrils and gloopy bits and amoebic pseudopodia keeping the ends together. There’s also a rather distressing smell, but that’s orthogonal to our discussion (thanks be to God). Now if you pull the bits apart far enough, the tendrils start to snap and you’re eventually left with two soggy pieces of bread rather than a single, disgusting whole.

The communication links within a team look like those tendrils of stale cheese. There’s a tremendous amount of communication which goes on within a team. You simply can’t stretch those links across a continent and expect them to hold together. You won’t end up with one team, but two distinct teams.

Now if you acknowledge that and formalise the interface between the two teams, then there is no problem. However, if you ignore the separation you’re going to end up facing all kinds of problems, all of which can be traced to the narrow communication bandwidth.

In short, stack two sandwiches one on top of the other if you want, but don’t try to pull one apart. That’s just nasty.

4. If analysis is incomplete, it’s a major red flag

Is analysis incomplete? Are the analysts walking around looking sheepish and mumbling something about Agile project management? Well, then, start screaming now and don’t stop till they bundle you away in the coat with the wrap around sleeves. Electro-shock therapy is positively relaxing compared to what you’re going to go through otherwise.

Not convinced? Then be prepared for a constant stream of ‘defects’ which are simply holes in the requirements and specifications. Watch the analysts dodge and deflect criticisms as they desperately try to blame everyone but themselves. Laugh as the client tears the hair off his head in frustration. Cry as you work week-ends to try and reduce the defect count. It’s a never ending spectacle of terror and rage!

Coming soon to a project near you! Well, unless you insist that the analysis be up to snuff before you start to build the application. I don’t care how much the client is screaming, complete the analysis. To do otherwise will simply postpone the whipping, not help you avoid it.

5. You can’t estimate use cases which have not been analysed yet

Seems pretty obvious, doesn’t it? So why oh why did the drivers of this train wreck go right ahead and do that? Why oh why did they then commit on these fantastical dates to the client? And why oh why did we not fall over ourselves laughing when they wanted us to meet those deadlines?

6. If the onshore team has never done offshore before, it’s a major project risk

There’s a first time for everything and ignorance is nothing to be ashamed of. However, not recognising the lack of experience as a project risk and not mitigating it is inexcusable. Do not pass ‘Go’, do not collect $200. It’s straight to jail for you.

7. One month RUP iterations are too short

A one month RUP iteration may be too short. The team was always running to keep up, with hardly any time to breathe. With a one month iteration, you have one week of defect fixing from the previous iteration, two weeks of build activity and then one week for integration. Of course, you’ve not actually planned for that one week of defect fixing, so everything is running behind schedule from the get go.

You have to choose a pace which is sustainable. Anything else is asking for developer frustration, burn-out and murderous rampages.

8. Requirements – keep them simple, keep them consolidated

If you find yourself with more than 3-5 requirement documents in your design document’s references section, then you’re in trouble. Scattering requirements across too many documents means you’ll always miss something out and have to re-work it.

Keep the analysts on a short leash. Left to themselves, they’ll loose themselves in a twisty maze of interdependent documents, all alike. Only it’s you who’s likely be eaten by a grue, not them.

9. Regular, formal communication is a must

If the project can afford it, then get yourselves a full or part time secretary. Involve him in all your onshore/offshore meetings and have him keep the minutes and publish them. Appointing someone from within the team might also work, but people hate to keep minutes.

You need at least two long video conferences a week to synchronise on design and one quick sync meeting a day for the team leads and testers. Keep them short.

10. Constant reviews are a must

You have to keep reviewing project and team performance on a background thread. Someone has to have enough mental bandwidth to spend some time every Nth iteration examining all the problems faced in N-1 and apply any lessons learnt to N+1. Don’t expect the project manager to do this, since he’ll be too busy fighting fires. Ideally, this should be done by the delivery head or the project champion.


So that’s it. The project is limping along with the harried offshore build team chained to the oars. Morale is low and the pace is slow. However, it’s still likely that the project will be delivered in some form or another. We’ll have to wait and watch.

The only bright spark is that the onshore team (which I’m holding responsible for much of the mess here) has had their bonuses canceled and are going to be getting coal in their stocking this year. Now that’s something to put a smile on Sadistic Santa’s face.

Ho. Ho. Ho.

Socialize: del.icio.us | digg | reddit | Technorati | Yahoo My Web

Development and Delivery or Management and Marketing Gimmicks?

SURGEON GENERAL’S WARNING: The following post contains extreme amounts of cynicism brought on by excessive contact with various marketroids and PHBs. Read at your own risk. May lead to hair-pulling, spontaneous gnashing of teeth and depression.

---

I’ve been working for a fairly typical, small IT services shop over the last year or so and I’ve seen projects come (rarely) and projects go (often). I’ve become increasingly frustrated by the insistence of upper management on concentrating on the wrong things, viz. management and marketing gimmickry rather than core delivery. I’ll be leaving this place in a couple of days (Hallelujah! :-), so here is an amalgam of what I’ve learnt about servicing your clients and knowing your markets (usually by watching us fail to do so properly).


At the lower end of the SME segment, the customer is more concerned about the actual delivery/success of the project while at the higher end, he's more concerned about the perception of delivery/success.


Let me illustrate what I mean by taking an extreme example. Let’s compare the actions and attitudes of a large trans-national IT services company (Big Corp) v/s a one man code shop (One Man Army a.k.a. Rambo). The actual situations below are not directly applicable except in the most extreme cases since you’ll no doubt be somewhere in the spectrum between Big Corp and Rambo, rather than at the ends of the rainbow. You’re relationship with your customer will thus be suitable nuanced.


The ‘customer’

Big Corp’s end customer is usually not the actual business entity they’re dealing with. It’s far more specific. It’s actually the business person they’re interacting with; the guy who’s brought them in. A ‘company’ has no mind of it’s own and despite the legal precedent set in many countries, it is not a ‘person’ in any sense of the word. It has no concept of profit or loss, or success or failure (or right and wrong for that matter). The people within it do.

This may seem like an extremely trite observation, but it is central to understanding the actions of any business. In any business relationship, the vendor/consultant/etc. will try and align themselves to the people they’re reporting to, not to the business at large. If an action will help their corporate patron succeed (and thus help them succeed) at the cost of the client business entity, they will do it

This is true for the one man shop too, but at the market level he operates within, he deals with people who are directly and immediately affected by the success or failure of the business as a whole. He’s usually dealing with the business owner or is only one step removed from him. He thus has to justify his presence by providing actual, immediate and tangible business benefits. He has to deliver.


The end goal

Now keeping in mind that the end customer is not the business but someone within it, the end goal shifts in a subtle way. It’s no longer necessarily about providing value, but the perception of value. Big Corp has to concentrate on providing its corporate patrons with ammunition they can use in the boardroom. They have to justify the expenses they’re incurring and to bolster their position. A project which is wildly successful, but cannot be used to further the position of the patron is useless compared to another which is a failure, but can be successfully spun as being otherwise.

The goal is to help the customer (i.e. Big Corps corporate sponsor) to move up the ladder, thus putting him in a position to reward Big Corp with larger and juicier contracts.

Rambo on the other hand, has to stay on the ball and actually deliver the product as specified. In a smaller business, any investment has to start showing returns pretty fast and a failure to do so can be catastrophic. Small business owners are infamous for trying to squeeze the maximum benefit out of every dime, since the dime in question usually belongs to the owner himself. The one man shop therefore cannot simply deal in perceptions, but has to provide tangible benefits in order to justify its presence. Its goal is thus to deliver the product and satisfy the customer (which is usually the actual business, not an employee within it) in order to receive more contracts in the future.


The ‘product’

The end product description too now undergoes a subtle change. The product is thus not necessarily working software - in time and under budget - but the perception of this having been delivered. Any roll-on effects on the business are usually so delayed that they can be safely ignored or blamed on other people later.

Rambo on the other hand, is stuck trying to actually deliver the product as specified.


The deliverable

Now we come to the meat of the matter. Having laid out the realities of life in the industry, what is the actual deliverable expected of the vendor and thus, what is it that is actually delivered.

Big Corp will concentrate on strengthening it’s corporate sponsor. Quick on-time delivery of the product is one way to do this, but it may be easier and cheaper to provide him with talking points, studies and white-papers instead. For example, Big Corp will spend quite some time giving it’s sponsors documents and presentations which will show just how profitable it is to farm business out to Big Corp like entities (but especially to Big Corp). Thus the deluge of studies favouring flavour-of-the-month management methodologies and out-sourcing techniques. A quick glance at the web site of any large IT vendor will show it to be replete with things of this nature.

The one man shop will concentrate on quick, on-time delivery. It’s very difficult for him to duck the bullet and find excuses for any defects. He has to deliver! Since he’s smaller, he doesn’t have a vast pool of potential victims clients to fall back on. Since the investment of the business in him is usually small as well, it’s easy to cut him adrift. Having relatively more to lose and few ways of weaseling out of his obligations, Rambo is usually forced to do a better job of things, or perish trying.


The end

So what does all this mean for smaller IT service shops? Simply this.

Cut the crap.

If you're a small company, you need to concentrate on development and delivery rather than trying to copy the big boys and talking about ‘management initiatives’, two-in-a-box hierarchies and all that jazz. Your customer is not interested in your marketing spiel because the market you’re trying to serve is extremely result oriented. Save the bullshit for when you’re all grown up.

Figure out how to deliver projects on time and under budget or be forced out of the market. It's as simple as that.

In fact, it's so simple and obvious, I just might have to write a white-paper on it...

Socialize:  del.icio.us | digg | Furl | reddit | Rojo | Spurl | Technorati | Yahoo My Web

Generation Gap

"The virtues required in military officers [in Second Generational militaries are] careful, even obsessive attention to process; avoiding risky decisions, and whenever possible making decisions by committee; avoiding responsibility; careerism, because success is measured by career progression; and generally shining up the handle on the big front door. Time is not very important, while dotting every i and crossing every t is vital, since at some point the auditors will be coming...As time goes on, efficiency tends to become more important than effectiveness...

...the virtues a Third Generation military requires in its officers [are different]. Those virtues—eagerness to make decisions and take responsibility, boldness, broad-mindedness and a spirit of intellectual inquiry, contempt for careerism and careerists—are not wanted in Second Generation militaries, and officers who demonstrate them are usually weeded out early. A Third Generation culture is difficult to maintain, and even more—impossible perhaps?—to restore once lost."

-- William S. Lind (Regression)

I’m a bit of an armchair general. It’s nothing more than a boyish fascination with things that clank around and go bang, but I do believe that the stratagems of war and the organisations which implement them have much to teach us. The battlefield is Darwinian by nature. If an idea has merit, then you live another day and if it’s DOA, then you’re KIA. In war, a lot of things are stripped to their core and stand naked in their essence. If an idea works on the battlefield, it’ll usually work just as well in a less lethal environment as well.

Now this isn't exactly a stunning new insight or anything. MBA types have been proudly lugging around copies of "The Art of War" since at least the 80's and perhaps longer. However, I believe it's not just overarching stratagems which might have some value, but the nitty-gritty of day to day organisation as well.

One area where there is a lot to be learnt, is in team organisation. An army deals with ‘projects’ which are under severe time, resource and quality pressures and with team members who are consequently under intense pressure as well. Ground realities change on a minute by minute basis and new challenges and opportunities pop up constantly. The quality of personnel is quite variable, which a few star performers and a whole load of dross, but everyone has to be put to work as best as possible. The overall techniques are well understood, but the devil is in the details of application. Hell, it sounds just like a software project! :-)


The Generation Gap

Military historians tend to segregate the organisational structure of fighting forces into ‘generations’:

1. First Generation armies/warfare: This was the first proper organisation of forces beyond a violent mob. It is completely obsolete and no longer observed. We’re talking lancers in formation and cavalry charges here.
2. Second Generation armies/warfare: This generation is geared towards wars of attrition where both the enemy and your own men are de-individualised and treated as numbers. It’s all about lobbing the maximum tonnage of shells at one another and slow, steady advances, with victory dependant more on industrial capacity than brilliant planning. The communication is largely top-down, with (one hopes) military savants on top directing every move on the ground far below. WW 1 is a good example of this, with static lines of defence and trench warfare.
3. Third Generation armies /warfare: This relies on surprise and speed, and depends far more on the quality of the individual soldier. Units are small and highly independent and attempt to get inside the decision loop of the opponent, acting faster than he can respond. The lines of communication are more bi-directional here, with significant input coming from below. In fact, most tactical decisions will probably be made at the lower levels directly. The German Blitzkrieg from WW2 is a classic example of this type of warfare.
4. Fourth Generation armies/warfare: This is war by decentralised, non-state actors against states, populations and other non-state actors. Units are very small and cell like, almost completely independent and ideologically committed; relying on the media as much as on force of arms to achieve their aims. The recent military and strategic victory of Hizbullah over Israel in 2006 is a stunning example of the effectiveness of this strategy, if properly implemented. It’s Third Generation warfare on steroids.
   

So what’s the difference in levels of effectiveness? Let’s compare 3G to 2G first. The Wehrmacht in WW2 was certainly 3G and performed very well against the rest of Europe. German tanks cut through the Maginot Line with ease and swiftly defeated every army they faced. It required the combined resources of two continents (Most of Asia and Northern America) and several disastrous decisions on the part of the German high command to finally roll them back. When Allied forces war-gamed some of the battles after the war, they found that it required 25% more Allied troops to equal the performance of the Germans and this was attributed almost entirely to the high quality of the officer corps (in other words, their application of 3G concepts which requires well-trained, independent troops).

Even more dramatic was the recent clash between a 4G and a 2G force, when Hizbullah defeated Israel (Israel was once a 3G force, but you don’t need to be a good soldier to massacre civilians and so the IDF has gradually withered in competence). By the last days of the war, Israel had fielded in excess of 40,000 troops, with artillery and air/naval support, while Hizbullah had only one light infantry brigade of around 3000 in the fight. They never felt the need to reinforce them. In other words, 3000 whipped 40,000. With a ratio of around 1:13, that means that Hizbullah was 13x or 1300% more effective than Israel. A truly staggering difference. Read this complete analysis for more


Software organisations

So what does all of this have to do with software projects? Well, my own observation is that software organisations can also be defined in much the same ways as military organisations. If we segregate them by generations, just like we did with armies, we get:

1. First Generation organisations: The first groupings of programmers. Obsolete.
2. Second Generation organisations: Most service firms making bespoke software fall into this category. Projects are about size and scope, with managers trying to increase team size to the max. Developers are treated as cogs in a giant wheel; as perfectly replaceable components, to be swapped in and out as desired. It’s all about project plans, matrices and counting man-hours, a few PHBs attempting to control the entire project from above while the unwashed mill about below. It’s a war of attrition against the client and the goal, with rigid attention to rules and an emphasis on blind obedience.
3. Third Generation organisations: Most start-ups, especially those geared towards producing an application, fall into this category. Teams are small and the work fast-paced. Individuals care more about doing the job than looking like their doing the job. Leaders emerge almost spontaneously and eagerly accept new responsibilities. Innovation is encouraged and boldness is rewarded.
4. Fourth Generation organisations: I’m going to go out on a limb here and say that Free Software is probably Fourth Generational. You have widely distributed teams, amorphously organised and made up of disjointed individuals united only by ideology. Individuals and sub-teams join and leave almost at random, but the project still forges ahead under the leadership of a charismatic leader.
   

A comparison of how the various generations fare against one another is fairly straight-forward. Start-ups don't always succeed against entrenched players, but they're the ones who usually supply the surprise upsets and market changing products which unseat the Big Boys. Netscape, Google, Amazon and Apple are some names which come to mind. Companies like Google seem to instinctively realise that they must retain their 3G edge, even as they grow and we've seen a whole lot of very innovative ideas on project and people management come out of there.

It's the 4G actors which are complete wild-cards. A good Open Source (though it's Free Software which is really4G) can either open up new markets and platforms for commercial concerns or completely take over a segment and devastate existing players. Or both.

You can really go far with this analogy, co-relating the tactics and strategies of 4G armed forces with 3/4G software entities. Much can be gleaned from examining the successes of 3/4G actors against less evolved opponents. I might elaborate about that at a later time, since I have a truly marvelous post about that in mind, which this column is too narrow to contain.

Personal Considerations

There are few things which guarantee more frustration than being a 3G person in a 2G shop and there are few things more pathetic than a 2G person in a 3G team. The last part of the quote by Lind above, the bit about 3G personality types being weeded out early, is very relevant since it happens quite frequently. The person may not be physically removed, but his 3G characteristics are usually excised. He may not leave the company, but he will dampen down his native enthusiasm and vigour and fall in line with the rest of the drones; at least for a little while. It can’t last forever though and eventually, he will leave for more hospitable shores.

Organisations are not completely stagnant though. Small 3G organisations eventually grow larger and devolve into 2G ones, while large 2G behemoths may occasionally become 3G (or at least 2.5G) in times of crisis. This opens us new opportunities for dormant 3/2G types. You’ll see this happening all the time in armies. Generals who’ve done very well for themselves in peacetime armies are usually dead within months of the start of hostilities, killed either by the enemy or their own side. On the other hand, effective commanders who rise rapidly through the ranks during war are usually shunted aside once combat ends and it’s back to boardroom battles. History is replete with instances of charismatic Generals who’ve ended up committing professional or physical suicide once the war is over.


Some Parting Advice

My personal experience is that the generation gap between your own personality and that of the organisation you work for can be the source of a lot of mental and emotional stress. Trying to swim against the current is very, very exhausting and only very rarely worth the effort. If you’re a 2G type, then accept that and work in an organisation which rewards your particular leanings and if you’re a 3G or 4G type, then for God’s sake, avoid <3G organisations like the plague. You may strike it lucky and end up in a 3G team in a 2G world, but that’s usually a complete fluke. Just skip the frustration and head straight to where you’ll be appreciated. That’s my personal experience and for once, I’m going to be applying this bit of advice to my own personal life.

I can’t help it, this post's author just makes so much sense! :-D