Generation Gap

"The virtues required in military officers [in Second Generational militaries are] careful, even obsessive attention to process; avoiding risky decisions, and whenever possible making decisions by committee; avoiding responsibility; careerism, because success is measured by career progression; and generally shining up the handle on the big front door. Time is not very important, while dotting every i and crossing every t is vital, since at some point the auditors will be coming...As time goes on, efficiency tends to become more important than effectiveness...

...the virtues a Third Generation military requires in its officers [are different]. Those virtues—eagerness to make decisions and take responsibility, boldness, broad-mindedness and a spirit of intellectual inquiry, contempt for careerism and careerists—are not wanted in Second Generation militaries, and officers who demonstrate them are usually weeded out early. A Third Generation culture is difficult to maintain, and even more—impossible perhaps?—to restore once lost."

-- William S. Lind (Regression)

I’m a bit of an armchair general. It’s nothing more than a boyish fascination with things that clank around and go bang, but I do believe that the stratagems of war and the organisations which implement them have much to teach us. The battlefield is Darwinian by nature. If an idea has merit, then you live another day and if it’s DOA, then you’re KIA. In war, a lot of things are stripped to their core and stand naked in their essence. If an idea works on the battlefield, it’ll usually work just as well in a less lethal environment as well.

Now this isn't exactly a stunning new insight or anything. MBA types have been proudly lugging around copies of "The Art of War" since at least the 80's and perhaps longer. However, I believe it's not just overarching stratagems which might have some value, but the nitty-gritty of day to day organisation as well.

One area where there is a lot to be learnt, is in team organisation. An army deals with ‘projects’ which are under severe time, resource and quality pressures and with team members who are consequently under intense pressure as well. Ground realities change on a minute by minute basis and new challenges and opportunities pop up constantly. The quality of personnel is quite variable, which a few star performers and a whole load of dross, but everyone has to be put to work as best as possible. The overall techniques are well understood, but the devil is in the details of application. Hell, it sounds just like a software project! :-)


The Generation Gap

Military historians tend to segregate the organisational structure of fighting forces into ‘generations’:

1. First Generation armies/warfare: This was the first proper organisation of forces beyond a violent mob. It is completely obsolete and no longer observed. We’re talking lancers in formation and cavalry charges here.
2. Second Generation armies/warfare: This generation is geared towards wars of attrition where both the enemy and your own men are de-individualised and treated as numbers. It’s all about lobbing the maximum tonnage of shells at one another and slow, steady advances, with victory dependant more on industrial capacity than brilliant planning. The communication is largely top-down, with (one hopes) military savants on top directing every move on the ground far below. WW 1 is a good example of this, with static lines of defence and trench warfare.
3. Third Generation armies /warfare: This relies on surprise and speed, and depends far more on the quality of the individual soldier. Units are small and highly independent and attempt to get inside the decision loop of the opponent, acting faster than he can respond. The lines of communication are more bi-directional here, with significant input coming from below. In fact, most tactical decisions will probably be made at the lower levels directly. The German Blitzkrieg from WW2 is a classic example of this type of warfare.
4. Fourth Generation armies/warfare: This is war by decentralised, non-state actors against states, populations and other non-state actors. Units are very small and cell like, almost completely independent and ideologically committed; relying on the media as much as on force of arms to achieve their aims. The recent military and strategic victory of Hizbullah over Israel in 2006 is a stunning example of the effectiveness of this strategy, if properly implemented. It’s Third Generation warfare on steroids.
   

So what’s the difference in levels of effectiveness? Let’s compare 3G to 2G first. The Wehrmacht in WW2 was certainly 3G and performed very well against the rest of Europe. German tanks cut through the Maginot Line with ease and swiftly defeated every army they faced. It required the combined resources of two continents (Most of Asia and Northern America) and several disastrous decisions on the part of the German high command to finally roll them back. When Allied forces war-gamed some of the battles after the war, they found that it required 25% more Allied troops to equal the performance of the Germans and this was attributed almost entirely to the high quality of the officer corps (in other words, their application of 3G concepts which requires well-trained, independent troops).

Even more dramatic was the recent clash between a 4G and a 2G force, when Hizbullah defeated Israel (Israel was once a 3G force, but you don’t need to be a good soldier to massacre civilians and so the IDF has gradually withered in competence). By the last days of the war, Israel had fielded in excess of 40,000 troops, with artillery and air/naval support, while Hizbullah had only one light infantry brigade of around 3000 in the fight. They never felt the need to reinforce them. In other words, 3000 whipped 40,000. With a ratio of around 1:13, that means that Hizbullah was 13x or 1300% more effective than Israel. A truly staggering difference. Read this complete analysis for more


Software organisations

So what does all of this have to do with software projects? Well, my own observation is that software organisations can also be defined in much the same ways as military organisations. If we segregate them by generations, just like we did with armies, we get:

1. First Generation organisations: The first groupings of programmers. Obsolete.
2. Second Generation organisations: Most service firms making bespoke software fall into this category. Projects are about size and scope, with managers trying to increase team size to the max. Developers are treated as cogs in a giant wheel; as perfectly replaceable components, to be swapped in and out as desired. It’s all about project plans, matrices and counting man-hours, a few PHBs attempting to control the entire project from above while the unwashed mill about below. It’s a war of attrition against the client and the goal, with rigid attention to rules and an emphasis on blind obedience.
3. Third Generation organisations: Most start-ups, especially those geared towards producing an application, fall into this category. Teams are small and the work fast-paced. Individuals care more about doing the job than looking like their doing the job. Leaders emerge almost spontaneously and eagerly accept new responsibilities. Innovation is encouraged and boldness is rewarded.
4. Fourth Generation organisations: I’m going to go out on a limb here and say that Free Software is probably Fourth Generational. You have widely distributed teams, amorphously organised and made up of disjointed individuals united only by ideology. Individuals and sub-teams join and leave almost at random, but the project still forges ahead under the leadership of a charismatic leader.
   

A comparison of how the various generations fare against one another is fairly straight-forward. Start-ups don't always succeed against entrenched players, but they're the ones who usually supply the surprise upsets and market changing products which unseat the Big Boys. Netscape, Google, Amazon and Apple are some names which come to mind. Companies like Google seem to instinctively realise that they must retain their 3G edge, even as they grow and we've seen a whole lot of very innovative ideas on project and people management come out of there.

It's the 4G actors which are complete wild-cards. A good Open Source (though it's Free Software which is really4G) can either open up new markets and platforms for commercial concerns or completely take over a segment and devastate existing players. Or both.

You can really go far with this analogy, co-relating the tactics and strategies of 4G armed forces with 3/4G software entities. Much can be gleaned from examining the successes of 3/4G actors against less evolved opponents. I might elaborate about that at a later time, since I have a truly marvelous post about that in mind, which this column is too narrow to contain.

Personal Considerations

There are few things which guarantee more frustration than being a 3G person in a 2G shop and there are few things more pathetic than a 2G person in a 3G team. The last part of the quote by Lind above, the bit about 3G personality types being weeded out early, is very relevant since it happens quite frequently. The person may not be physically removed, but his 3G characteristics are usually excised. He may not leave the company, but he will dampen down his native enthusiasm and vigour and fall in line with the rest of the drones; at least for a little while. It can’t last forever though and eventually, he will leave for more hospitable shores.

Organisations are not completely stagnant though. Small 3G organisations eventually grow larger and devolve into 2G ones, while large 2G behemoths may occasionally become 3G (or at least 2.5G) in times of crisis. This opens us new opportunities for dormant 3/2G types. You’ll see this happening all the time in armies. Generals who’ve done very well for themselves in peacetime armies are usually dead within months of the start of hostilities, killed either by the enemy or their own side. On the other hand, effective commanders who rise rapidly through the ranks during war are usually shunted aside once combat ends and it’s back to boardroom battles. History is replete with instances of charismatic Generals who’ve ended up committing professional or physical suicide once the war is over.


Some Parting Advice

My personal experience is that the generation gap between your own personality and that of the organisation you work for can be the source of a lot of mental and emotional stress. Trying to swim against the current is very, very exhausting and only very rarely worth the effort. If you’re a 2G type, then accept that and work in an organisation which rewards your particular leanings and if you’re a 3G or 4G type, then for God’s sake, avoid <3G organisations like the plague. You may strike it lucky and end up in a 3G team in a 2G world, but that’s usually a complete fluke. Just skip the frustration and head straight to where you’ll be appreciated. That’s my personal experience and for once, I’m going to be applying this bit of advice to my own personal life.

I can’t help it, this post's author just makes so much sense! :-D

On the business of software

A long, but very accurate quote from Joel Spolsky. I completely agree with both the points made by him. I wish I had more to say than that, but there really isn’t much else to add… :-)

"The number one thing is a micro-ISV shouldn't be one person, it should be two people at the very least and one of them should have the business and marketing and sales skills experience. The other one should have the tech skills and the programming and the inventing the product type of skills. That kind of partnership is far more likely to be successful than the individual working alone just because people don't usually have both of those skill sets and so they really need to all be covered.

Or if you have only the sales and marketing you're not going to be to be successful because you won't have a good product and if you don't have product development skills you won't be successful because no one will hear about you and the business side won't really work. Having two people, I feel, is crucial just to validate your idea, almost to keep each other motivated, bounce ideas off each other and so on - that sort of thing.

The first part is the minimum size for a micro-ISV that can go anywhere beyond a fun project.

The second part, and Bob alluded to this earlier, which is my prototypical example of the photo gallery which is probably nine million micro-ISVs have made an application where it's like "Hey, everybody's got these digital cameras my application lets you upload all your pictures and put them on the web and make web galleries." There have been about a million of these and a very tiny number of them have been successful and the vast majority of them have been instant flops. For some reason this is an incredibly appealing idea for software developers to do, maybe because they feel like they know how to do everything, all the steps they're going to need to do to write the code to make this work, but for some reason they never really make it work.

But what I've always told these people time and time again, and they never listen to me, is instead of making the generic "upload your pictures application" take a very, very small niche audience - wedding photographers - and make the ultimate application for wedding photographers. Find out exactly what wedding photographers need. There's a lot of money around wedding photographers, they get paid an awful lot of money, and figure out exactly what their workflow is. If you need to find wedding photographers because they're in the yellow pages and there are directories of these things. Call them all and find out what they want and try to sell them your solution.

And so what I always tell micro-ISVs is, and that's just an illustrative picture, try to narrow your potential audience almost as much as possible to get started. In order to bootstrap you're going to have to find a very small initial audience you can serve extremely well of people who all speak to each other. One you can find all in one place, where there's money being spent because you're going to need to get a part of it for this thing to work. And once you find that very narrow niche, that's the way you get bootstrapped and you can think about crossing the chasm as Jeffrey Morris says into other kinds of industries and other kinds of larger markets. But you really need to pick something vertical to start with."

Read the rest at The MicroISV Show #10 - Joel Spolsky

Codephobia

Grady Booch had come to Manchester for a couple of days last month and I’d gone for his talk about "The promise, The limits, The beauty of Software". There were several interesting themes explored, but it was one comment made by him, almost in passing, which really struck a chord with me

He mentioned how he’s seen organisations which seem as if they’re afraid of code and that jibes with what I’ve noticed as well. An organisation or project or team which is completely tied up with process, to the extent that almost nothing can be done without reams of paper being produced, is usually a victim of codephobia. Before a single line can be written or modified, several Word documents will have to be either produced or updated, items in the project tracking system will have to be massaged, meetings will have to be called, recalcitrant team leads will have to cudgelled into submission and developers will have to be dragged to their seats – their nails leaving deep groves in the carpeting as they go.

Now mind you, this isn’t the usual screed by frustrated programmers against the horrors of ‘Process’. The ability to track, managed and control a project is vital to it’s success. However, smart, successful firms realise that all of these things are a means to an end. The customer wants a working application, not reams of design documents and lists of tracked defects. All of these are meant to enable you to write code better and faster.

This simple concept however, seems to be beyond the ability of many people. They act as if the various artifacts of process adherence were what the customer was paying them for and not the actual application. At heart, it’s all about being afraid of putting pen to paper - of producing the code - because that’s when the inadequacies will start to show and the incompetence will start to bubble to the surface, like marsh gas from a bog. In other cases, it’s the managers fear of the unknown. Most project managers are code-illiterate and so fear what they cannot understand or accurately track. They understand documents though and are happy to lose themselves in them. They’d love to have people code in Word if they could. Notice the popularity of code generators in such teams. It’s a dead give away :-)

In any successful team that I’ve seen, the focus is completely on the code. Everything is subservient to it and team is made up of and lead by competent programmers. Only a minimum amount of ‘process’ is tolerated and it’s all about knuckling down and churning out working code as soon as possible. Codephobic teams postpone code generation to the last moment and have to be dragged to their compilers kicking and screaming…

Which is kind of poetically just, because that is exactly what their customers end up doing when they see the end result.

Because the stakes are so small

"University politics are vicious precisely because the stakes are so small."
-- Henry Kissinger

I’m not a fan on Henry Kissinger by any means, but even the Devil can be right sometimes. It’s not university politics which concerns me though, but the sometimes byzantine intrigue which takes place within even relatively minor projects.

I’ve found office politics to be in turns, disgusting, depressing and distressing. It can occasionally be amusing, but only in the slightly tight-lipped, pathetic way an argument between two drunken bums might be amusing. What it usually is, is soul-destroyingly depressing. It sours the atmosphere of the entire project, splits teams and causes you to loose respect for people you’ll probably have to deal with on a daily basis.

And it’s usually so petty. The motivations are so base; jealousy, imagined slights, over-reaching ambition and profound insecurities. The tactics are so filthy; lies, rumour mongering and gossip. And what are the fruits? The imagined goals are usually minor things like a promotion, a slightly better appraisal or even something as petty as sucking up to management. The eventual result is usually bitterness, a loss of respect of others for ones-self and the debasement of your own soul.

How is one to deal with it when it starts? I have no clue. The people who are of this mould are usually incorrigibly bent out of true. You can try to straighten them out or bend yourself to fit their twisted psyche, but it’s usually futile. Both your resistance or compliance will trigger something within them which will make you their target. The base (and it is very base) cause, is that deep down, they actually enjoy the cut and thrust of it. And that won’t change until they do.

So should one get down in the mud and wrestle with them? Nope. You’ll feel disgusted with yourself, while they’ll be loving it. The best policy is to keep your interaction with such people to a minimum and just get on with doing a good job. They’ll either make so many enemies, they’ll finally be pushed out…

…or they’ll become the CEO.