10 interesting lessons from 1 interesting project

I’ve spent six interesting months on my current project and my association with it is at last coming to a close. The design phase is largely over and I can finally move on to other things.

Like some sort of technological Santa Clause, I skip from chimney to chimney spreading good cheer, pixie dust and sooty footprints as I go. I like to think I leave happy families in my wake, opening the little ‘presents’ I’ve left behind to shrieks of excitement (or is that terror? I never can tell). I do know they’ll be looking forward to seeing me once more, no matter how much coal they ended up with in their stocking this year :-).

Anyway, this current assignment has been a most interesting project; as in “May you live in interesting times” and “Why yes, Nurse, this is quite an interesting case”.

Yup, that kind of interesting.

It’s a pity that most of the lessons that this project have taught me are of the political, managerial and interpersonal variety than those of the technical type! I’ve noted some of them down in this post, more as a reminder to myself than anything else.

A quick overview


The project aims to develop a Java based data entry application for a large Swiss financial services firm. The onshore team was from a German company (part of the corporate group) and the offshore development was handled in Mumbai. There are around 20 devs, 5 designers and various testers, managers and other assorted mischief makers. The project is now over budget, delayed and the quality of the deliverable is not up to snuff. The only reason there’s any hope of completing in time is because individually, the team members are very highly skilled. It’s just that getting people to work together seems to be very difficult.

I joined the project just six months ago, when the good ship INS T******s had already struck ice and was beginning to list rather alarmingly. I came onto the scene just in time to miss the preceding bit where most of the mistakes had been made, but just in time to start seeing their effects start to become apparent. It’s no fun joining a project and immediately realising that you’re going to have to fight your way to the lifeboats before long! :-)

Mind you, it’s not been as bad as some projects I’ve worked on, but in many ways it’s been even more frustrating, because success was so close, and now it’s oh so far.


The 10 lessons

1. People have to be aligned to project success

If people can succeed at the expense of the project, then the project will suffer. If you make it almost necessary for them to bury the project to further themselves, then stand clear of the stampede for the shovels!

People are not intrinsically evil. They want to do the right thing and they will too, if given the chance. However, if you inadvertently stack the environment such that the only way they can advance their own position is by stepping on someone else’s face, then there will be a lot of people wandering around with bloodied mugs. It’s the responsibility of the project manager(s) to align individuals and teams properly, so everyone is pulling together towards project completion and success.

Fail to do so and you doom the project to delay and possible failure.

Some things to keep in mind:

1. There should be no gain in shunting blame between onshore and offshore.
2. Onshore should not be allowed to ‘succeed’ by leaving offshore behind and vice-versa.
3. Onshore should gain nothing by show-casing itself independently before the client and neither should offshore.
4. Onshore should be reassured that their roles and jobs are secure and that they will not be ‘Bangalored’.
5. Offshore should not attempt to aggressively infiltrate and displace onshore. See point 4.

2. Architectural Vision is vital

You don’t have to nail down each and every detail before you start the build. You can back-track once the build has started and update the design to reflect new learning’s. However, you cannot start the build without having a very clear picture of what you want, how you want it and how you’re going to get it.

The first thing I noticed and flagged when I joined the project was a lack of project vision. No one really knew how exactly things were going to interact with each other or how changes to one component were going to impact another. There were a lot of things which were just up in the air and it was anyone’s guess as to where they were going to land.

A major reason behind this was the lack of a proper POC (proof of concept). Had one been constructed, the entire design and build stages would have been several orders of magnitude easier. If there’s one thing I’ve taken away from this project, it’s this:

Don’t skip the POC.

3. Never split tasks and teams across geographies

Take a stale, grilled cheese sandwich and start pulling it apart. You’ll notice that there’re all kinds of tendrils and gloopy bits and amoebic pseudopodia keeping the ends together. There’s also a rather distressing smell, but that’s orthogonal to our discussion (thanks be to God). Now if you pull the bits apart far enough, the tendrils start to snap and you’re eventually left with two soggy pieces of bread rather than a single, disgusting whole.

The communication links within a team look like those tendrils of stale cheese. There’s a tremendous amount of communication which goes on within a team. You simply can’t stretch those links across a continent and expect them to hold together. You won’t end up with one team, but two distinct teams.

Now if you acknowledge that and formalise the interface between the two teams, then there is no problem. However, if you ignore the separation you’re going to end up facing all kinds of problems, all of which can be traced to the narrow communication bandwidth.

In short, stack two sandwiches one on top of the other if you want, but don’t try to pull one apart. That’s just nasty.

4. If analysis is incomplete, it’s a major red flag

Is analysis incomplete? Are the analysts walking around looking sheepish and mumbling something about Agile project management? Well, then, start screaming now and don’t stop till they bundle you away in the coat with the wrap around sleeves. Electro-shock therapy is positively relaxing compared to what you’re going to go through otherwise.

Not convinced? Then be prepared for a constant stream of ‘defects’ which are simply holes in the requirements and specifications. Watch the analysts dodge and deflect criticisms as they desperately try to blame everyone but themselves. Laugh as the client tears the hair off his head in frustration. Cry as you work week-ends to try and reduce the defect count. It’s a never ending spectacle of terror and rage!

Coming soon to a project near you! Well, unless you insist that the analysis be up to snuff before you start to build the application. I don’t care how much the client is screaming, complete the analysis. To do otherwise will simply postpone the whipping, not help you avoid it.

5. You can’t estimate use cases which have not been analysed yet

Seems pretty obvious, doesn’t it? So why oh why did the drivers of this train wreck go right ahead and do that? Why oh why did they then commit on these fantastical dates to the client? And why oh why did we not fall over ourselves laughing when they wanted us to meet those deadlines?

6. If the onshore team has never done offshore before, it’s a major project risk

There’s a first time for everything and ignorance is nothing to be ashamed of. However, not recognising the lack of experience as a project risk and not mitigating it is inexcusable. Do not pass ‘Go’, do not collect $200. It’s straight to jail for you.

7. One month RUP iterations are too short

A one month RUP iteration may be too short. The team was always running to keep up, with hardly any time to breathe. With a one month iteration, you have one week of defect fixing from the previous iteration, two weeks of build activity and then one week for integration. Of course, you’ve not actually planned for that one week of defect fixing, so everything is running behind schedule from the get go.

You have to choose a pace which is sustainable. Anything else is asking for developer frustration, burn-out and murderous rampages.

8. Requirements – keep them simple, keep them consolidated

If you find yourself with more than 3-5 requirement documents in your design document’s references section, then you’re in trouble. Scattering requirements across too many documents means you’ll always miss something out and have to re-work it.

Keep the analysts on a short leash. Left to themselves, they’ll loose themselves in a twisty maze of interdependent documents, all alike. Only it’s you who’s likely be eaten by a grue, not them.

9. Regular, formal communication is a must

If the project can afford it, then get yourselves a full or part time secretary. Involve him in all your onshore/offshore meetings and have him keep the minutes and publish them. Appointing someone from within the team might also work, but people hate to keep minutes.

You need at least two long video conferences a week to synchronise on design and one quick sync meeting a day for the team leads and testers. Keep them short.

10. Constant reviews are a must

You have to keep reviewing project and team performance on a background thread. Someone has to have enough mental bandwidth to spend some time every Nth iteration examining all the problems faced in N-1 and apply any lessons learnt to N+1. Don’t expect the project manager to do this, since he’ll be too busy fighting fires. Ideally, this should be done by the delivery head or the project champion.


So that’s it. The project is limping along with the harried offshore build team chained to the oars. Morale is low and the pace is slow. However, it’s still likely that the project will be delivered in some form or another. We’ll have to wait and watch.

The only bright spark is that the onshore team (which I’m holding responsible for much of the mess here) has had their bonuses canceled and are going to be getting coal in their stocking this year. Now that’s something to put a smile on Sadistic Santa’s face.

Ho. Ho. Ho.

Socialize: del.icio.us | digg | reddit | Technorati | Yahoo My Web

Borged!

I’ve finally started blogging again after a long hiatus, but this time it’s from the belly of the beast. I’ve been swallowed by the Borg. Yes indeed, I’ve abandoned my earlier aversion to formal pants servile ties and joined a large, faceless multi-national.

Due to the vagaries of chance, I’ve always ended up in smaller companies before this. Things just turned out that way. Anyway, I’d always suspected that I might have a bit of difficulty fitting into a more formal corporate culture. Dilbert (the true guide to corporate culture everywhere!) didn’t paint too rosy a picture either :-) But surprise, surprise, it’s not so bad (he squealed, as he hung by his thumbs), it’s not so bad at all.

Some quick points of comparison:

1. The Facilities: Most smaller companies are unable to offer amenities like a bus service, a proper canteen, housing or other sorts of facilities. These may seem like little things, but they’re important hygiene factors. And speaking of hygiene, you finally get clean toilets! :-)
2. Training: This is the first place I’ve been in which offers compulsory training; 40 hours at a minimum every year. More if you can swing it. Now, I’m not a fan of structured, formal learning (having been processed by enough educational institutions in the past), but it’s not a complete waste of time either.
3. The Impersonality: Brilliant! I’m finally anonymous! :-) It feels great to be a little fish in a big pond, instead of things being the other way around. Now, this will surprise many people, but it’s actually tiring (and then eventually irritating) when everyone from the janitor to the CEO knows your name, face and birthmarks. This is probably the introvert in me talking, but there’s something to be said for being able to walk through the entire building and not have to constantly ‘Hi!’ everyone you see.
4. A larger pool of potential friends: Now for the extrovert in me :-) In an organization of several tens of thousands of wage slaves, you’re bound to come across malcontent deviants as insane as you are. What’s that you said? So you're a fan of red staplers, coke bottle glasses and naked flames too? Brilliant ! :-)
5. Openness: Paradoxically, larger companies are more open. Try and get the owner of a SME to divulge the details about the companies finances or its future direction. You'll have to break out the pliers to get anything close to the truth. Public corporations are legally obliged to reveal their financial details and you can read about the companies prospects in the daily broadsheet.
6. Permanence: The chances of you coming to work one fine day and finding a padlock on the door and shit-eating grin on your 'bosses' face are quite remote.
7. Multiple escalation paths: Got a problem with your superior in a smaller company? Tough luck, he’s probably the owner (or a close relative – hello nepotism!). Either live with him or leave. In large organizations, there’s usually a structured escalation path and conflict resolution forums. It’s much easier to iron things out.
8. Professionalism: The people you’re working with were (theoretically/presumably) hired because they are the best fit for the job, not because they’re the owner’s retarded cousin from Jalgaon or a member of his community.
9. Varied Projects: You don’t need to switch companies to switch projects.
10. Brand Recognition: Having to explain what your company does every time gets real old, real fast.
11. Better Opportunities for Travel: You don’t get many big spenders contracting with smaller companies. If you want to travel a bit, it’s better to join a larger firm.
12. Better long term prospects: Finally, room to grow! Larger firms have more space for growth, bother vertically and horizontally. In smaller organizations, I’ve usually ended up in a position where the only way up is to kill the owner or marry his daughter (preferably both). It feels great to be able to contemplate a career plan which doesn’t include the liberal use of rat poison to open up some space in the ranks first.
    

So what’s the reason behind the differences? It’s all pretty straight-forward: money and scale. And one more thing (echo’s of which can be found in this post). Larger organizations are run by employees in part for their own benefit and not the benefit of nebulous, anonymous ‘shareholders’. It’s not fair to the actual owners (a.k.a shareholders, a.k.a. chumps), but it’s a fact.

“What’s that you say, the team’s feeling ‘stressed’? Well, it’s off to Lonavla for a week-end then!” – Manager in Faceless Multinational Pvt. Ltd.

In smaller companies, the owner’s breathing down your neck every time you fill in an expense account.

“What’s that you say, the team’s feeling ‘stressed’? Well, call them in on the week-end for a side project I’ve been thinking about, that’ll cheer them up!” – Hari Sadu (SME CEO)

Maybe the only reason I’m enjoying myself here is because of ye olde “Herbage be greener on the other side”. Or maybe things really are better over here. Time will tell. But as I contemplate by assimilation into the beast, I have to accept that the Borg were right after all.

Resistance was futile.

Socialize: del.icio.us | digg | reddit | Technorati | Yahoo My Web

Development and Delivery or Management and Marketing Gimmicks?

SURGEON GENERAL’S WARNING: The following post contains extreme amounts of cynicism brought on by excessive contact with various marketroids and PHBs. Read at your own risk. May lead to hair-pulling, spontaneous gnashing of teeth and depression.

---

I’ve been working for a fairly typical, small IT services shop over the last year or so and I’ve seen projects come (rarely) and projects go (often). I’ve become increasingly frustrated by the insistence of upper management on concentrating on the wrong things, viz. management and marketing gimmickry rather than core delivery. I’ll be leaving this place in a couple of days (Hallelujah! :-), so here is an amalgam of what I’ve learnt about servicing your clients and knowing your markets (usually by watching us fail to do so properly).


At the lower end of the SME segment, the customer is more concerned about the actual delivery/success of the project while at the higher end, he's more concerned about the perception of delivery/success.


Let me illustrate what I mean by taking an extreme example. Let’s compare the actions and attitudes of a large trans-national IT services company (Big Corp) v/s a one man code shop (One Man Army a.k.a. Rambo). The actual situations below are not directly applicable except in the most extreme cases since you’ll no doubt be somewhere in the spectrum between Big Corp and Rambo, rather than at the ends of the rainbow. You’re relationship with your customer will thus be suitable nuanced.


The ‘customer’

Big Corp’s end customer is usually not the actual business entity they’re dealing with. It’s far more specific. It’s actually the business person they’re interacting with; the guy who’s brought them in. A ‘company’ has no mind of it’s own and despite the legal precedent set in many countries, it is not a ‘person’ in any sense of the word. It has no concept of profit or loss, or success or failure (or right and wrong for that matter). The people within it do.

This may seem like an extremely trite observation, but it is central to understanding the actions of any business. In any business relationship, the vendor/consultant/etc. will try and align themselves to the people they’re reporting to, not to the business at large. If an action will help their corporate patron succeed (and thus help them succeed) at the cost of the client business entity, they will do it

This is true for the one man shop too, but at the market level he operates within, he deals with people who are directly and immediately affected by the success or failure of the business as a whole. He’s usually dealing with the business owner or is only one step removed from him. He thus has to justify his presence by providing actual, immediate and tangible business benefits. He has to deliver.


The end goal

Now keeping in mind that the end customer is not the business but someone within it, the end goal shifts in a subtle way. It’s no longer necessarily about providing value, but the perception of value. Big Corp has to concentrate on providing its corporate patrons with ammunition they can use in the boardroom. They have to justify the expenses they’re incurring and to bolster their position. A project which is wildly successful, but cannot be used to further the position of the patron is useless compared to another which is a failure, but can be successfully spun as being otherwise.

The goal is to help the customer (i.e. Big Corps corporate sponsor) to move up the ladder, thus putting him in a position to reward Big Corp with larger and juicier contracts.

Rambo on the other hand, has to stay on the ball and actually deliver the product as specified. In a smaller business, any investment has to start showing returns pretty fast and a failure to do so can be catastrophic. Small business owners are infamous for trying to squeeze the maximum benefit out of every dime, since the dime in question usually belongs to the owner himself. The one man shop therefore cannot simply deal in perceptions, but has to provide tangible benefits in order to justify its presence. Its goal is thus to deliver the product and satisfy the customer (which is usually the actual business, not an employee within it) in order to receive more contracts in the future.


The ‘product’

The end product description too now undergoes a subtle change. The product is thus not necessarily working software - in time and under budget - but the perception of this having been delivered. Any roll-on effects on the business are usually so delayed that they can be safely ignored or blamed on other people later.

Rambo on the other hand, is stuck trying to actually deliver the product as specified.


The deliverable

Now we come to the meat of the matter. Having laid out the realities of life in the industry, what is the actual deliverable expected of the vendor and thus, what is it that is actually delivered.

Big Corp will concentrate on strengthening it’s corporate sponsor. Quick on-time delivery of the product is one way to do this, but it may be easier and cheaper to provide him with talking points, studies and white-papers instead. For example, Big Corp will spend quite some time giving it’s sponsors documents and presentations which will show just how profitable it is to farm business out to Big Corp like entities (but especially to Big Corp). Thus the deluge of studies favouring flavour-of-the-month management methodologies and out-sourcing techniques. A quick glance at the web site of any large IT vendor will show it to be replete with things of this nature.

The one man shop will concentrate on quick, on-time delivery. It’s very difficult for him to duck the bullet and find excuses for any defects. He has to deliver! Since he’s smaller, he doesn’t have a vast pool of potential victims clients to fall back on. Since the investment of the business in him is usually small as well, it’s easy to cut him adrift. Having relatively more to lose and few ways of weaseling out of his obligations, Rambo is usually forced to do a better job of things, or perish trying.


The end

So what does all this mean for smaller IT service shops? Simply this.

Cut the crap.

If you're a small company, you need to concentrate on development and delivery rather than trying to copy the big boys and talking about ‘management initiatives’, two-in-a-box hierarchies and all that jazz. Your customer is not interested in your marketing spiel because the market you’re trying to serve is extremely result oriented. Save the bullshit for when you’re all grown up.

Figure out how to deliver projects on time and under budget or be forced out of the market. It's as simple as that.

In fact, it's so simple and obvious, I just might have to write a white-paper on it...

Socialize:  del.icio.us | digg | Furl | reddit | Rojo | Spurl | Technorati | Yahoo My Web

Zen and the Art of Functional Programming

I’m a regular at http://programming.reddit.com and just about every day there’s at least one (or two or three) posts about some functional programming language or the other. Usually it’s Haskell which is being showcased, though you’ll also find entries for Erlang and occasionally for some of the more obscure ones like O’Caml or Dylan. I finally zoned out under the constant barrage of propaganda and moaning softly (“brains! braaaaiiiiins!) zombie-shuffled over to check out what the fuss was all about.

What’s all this about Zen then?

The Zen of programming is when you internalise the correct way of solving a problem using the programming model you’re working with. You might start off programming with something like C, using it in a classic procedural fashion. When you move from Procedural Programming on to something like Object Oriented Programming (OOP) using Java or C++ etc., it’s a bit of a culture shock at first. You keep trying to program procedurally, fighting the language every step of the way. There finally comes a moment however, when all the abstract concepts behind OOP fall into place with an almost audible snap and suddenly, in a moment of epiphany, you attain OOP enlightenment.

Now I can’t say I’ve achieve that level of union with the Tao of Functional Programming, but I am starting to finally grok what the whole thing is about. However, all opinions listed below are subject to rather radical change!

What’s Functional Programming (FP)?

I’ll save me some time and copy-paste in some definitions.

Functional programming is a style of programming that emphasizes the evaluation of expressions, rather than execution of commands. The expressions in these language are formed by using functions to combine basic values. A functional language is a language that supports and encourages programming in a functional style.

-- FAQ for comp.lang.functional

Functional programming is a programming paradigm that treats computation as the evaluation of mathematical functions and avoids state and mutable data. It emphasizes the application of functions, in contrast with the imperative programming style that emphasizes changes in state.

-- Wikipedia.

As far as I can make out so far, FP is all about decomposing the problem down to the various algorithms and data transformations involved and then cleanly enumerating them. We don’t really need to worry about the ‘objects’ involved (there aren’t any), or their relationship with one another. It’s all about dividing and conquering the problem by slicing it up into small bits and then figuring out what we need to do to each of the slices. Rather than identify objects, couple the data with behaviour and then deal with those objects, we pass both data and code around as required (and as easily), assembling and disassembling relationships as required.

Mind you, this is pretty much the SOP for most programming tasks, it’s just that FP makes it easier to work in this fashion. Besides, if you un-gag the Lispers for a moment, they will endlessly lecture you about the benefits of having small bits of code operate on large blobs of data. Rather like a shoal of Piranha reducing a buffalo. Be sure you re-gag them once you’re through or else they’ll never shut up.

Why spend time learning Functional Programming?

My primary reasons were:

1. A New Paradigm: Learning a new programming paradigm helps to stretch your mind and makes you a better programmer overall, even if you never directly apply any of the new techniques you’ve used. Notice I’m talking about paradigms here, not languages. Learning C++ if you know Java may make you a lightly better programmer over all, but going from Procedural programming to OOP for example, can be a mind-bending experience.
2. The Next Big Thing: The Functional programming weenies just can’t stop babbling about how they’re going to take over the world. And who knows, they just might. It takes a while for this sort of momentum to build up (look how long it took OOP to become mainstream!) but the signs abound; FP just might be the next big thing.
3. Increased Productivity: Anecdotal evidence suggests that writing code in a functional way leads to smaller, cleaner code and fewer bugs. The logic of the algorithm is clearly detailed and many of the internal details of the operations are hidden away, leading to less clutter. I’ve often heard that FP is about describing what you want done, while Imperative Programming ends up detailing how you want it done as well.
4. It’s Popping Up Everywhere!: FP isn’t all that obscure any more. You’ll regularly come across bits of functional code in various decidedly non FP languages. Java’s Comparator Interface is a good example of a common FP idiom, where you pass along a bit of code as a parameter as well as the data and have the code act on the data. Java 7 might have closures as well, which are already to be found in Ruby. Python has things like list comprehension, which has a distinctly FP feel and so on. I thought I might as well experience all these concepts in one integrated functional package rather than piecemeal.
   

FP vs OOP

Invariably when you start to talk about a new programming model, you’re met with slack jaws, blank stares and mewling cries of “But, but, but… OOP!”. Now I’m not one to deride OOP because I like the concept quite a lot. It’s a great fit for a range of applications and can really help model a lot of complex domains and interactions. However, it isn’t the end all and be all of computing that the OOP aficionados (and their groupies) make it out to be. One particular domain which seems to be a bit of a mismatch is the Web.

Most web programming involves a lot of pointless conversion from flat data to OOP and back again. Take a basic CRUD application. The user enters plain data without any OOP savvy behaviours or what have you. Just characters strung together. When he hits submit, we convert this data to fit into our OOP model and play around with it in the middle layer. We then flatten it once more and stick it in the database. We might also have some stored procedures in the database as well (usually written in a non-OOP language/manner). So the only place we’re really using OOP is in the middle layer and most of what we’re doing is simply converting data from one model to another. It’s just a huge waste of time. Embrace the fact that all we want to do is apply transformations to data and use FP (which is admirably suited to the role) to do just that.

I mean, what’s so Object Oriented about Servlets? They're actually very functional. You usually just implement one method/function which accepts user data as parameters, munges it and then forwards more data somewhere else, where we display the response.

Reuse?

As far as reuse goes, OOP hasn’t proven to be all that. Look at the Java Library. It’s just a bunch of methods. Useful one’s no doubt, but how many times have you inherited from one of it’s classes (other than Object of course! J)? You make an object which represents some data and you call methods on it. Most of the reusability is at the level of the methods. How much more useful then if we decouple the methods completely from the data, use duck typing and work on just about anything that’s passed in? Reusablity goes through the roof! Certainly, it won’t be any less than an OOP design.

Which FP Language should I go for?

“MMRRGH! (LISP!)” scream the Lispers through their gags, shouts of “Haskell!” from one corner and “Erlang!” from the other. They choice is endless. There are a whole load of functional languages out there, so which one should you pick up?

My 2p? Go for Erlang and Haskell. Both are relatively pure functional languages (unlike chimera like O’Caml, which are partly OOP and Lisp which is undifferentiated gloop and can be anything you want it to be) and fairly mainstream. Mind you, there’s nothing bad about being multi-paradigm, but when you’re learning FP, it’s best to suffer under a bit of disciple and be forced to be functional. Erlang is actually used in The Real World, though largely only in specialised hardware like network switches etc, which Haskell is under active developments and has a lot of mind-share.

I’m learning Erlang first because along with being FP, it has slightly different way of going about threading, though I hope to pick up Haskell as well (since it has another way of doing threading as well).

An Aside on Threading with Erlang and Haskell

Both Erlang and Haskell have taken a novel and (to me) refreshingly different way of going about threading. Neither concept is blindingly innovative, but it’s the first time I’ve seen it implemented as part of a language.

Threading is really coming into its own now. We’re moving into an age of massively multi-core CPUs where each core may be less powerful than the chips of today, but there’ll be so many per chip that the net speed will be tremendous. However, programs don’t multi-thread themselves. Developers have to identify concurrent paths and co-ordinate their interactions; something which can be quite difficult to code and fiendishly difficult to debug.

One advantage of FP is that functional code is inherently much easier to parallelise. Since functions don’t access or affect global state and act only on their parameters, you can infer possible parallel paths and have them run concurrently.

For e.g.

A = foo()
B = bar()

Here, the interpreter can run both foo() and bar()in parallel since they are not related and thus are guaranteed not to affect each other.

But wait, there’s more! Both Erlang and Haskell make threading even easier with innovative threading models. Rather than have process communicate through shared memory where you have to handle locking/concurrency yourself, both help make the process of writing multi-threaded code much easier.

1. Erlang (Actor/Message Model) : It’s Unix IPC all over again! You basically have pipes which you use to communicate between processes. Messages are asynchronous and not location specific, so processes can migrate to different machines transparently. Errors are piped to related processes as well, giving you robust error handling. No shared memory and scaling it trivial. In fact, the possibility of basically almost unlimited scaling across multiple machines is what really draws me to Erlang and has me drooling like an idiot. Write your code properly and you can just keep slotting in boxes. Erlang also has the ability to update code while it’s running, which means theoretically zero downtime if you’re careful. And it’s all actually in use in the telecommunications industry, so all this is real, not vapour.
2. Haskell (Software Transactional Memory Model) : Database transactions, but in local memory! Separate threads run within their own transactions and see a consistent view of the world. No need to explicitly lock bits of shared memory, we just let the system handle all the error prone bits and concentrate on our logic, confidently that our threads won’t be stepping on each other toes. This is a really powerful abstraction and it’s something that had me smacking my forehead, wondering why I didn’t think of it before. However, I see some fundamental problems with multi-machine scalability. This is a fairly well explored problem in the database world and I’m not sure I want my code doing two phase commits and roll-backs in a cluster. Still cool though.
   

Conclusions

There’s still a long way for me to go yet. I’m going to be going for a vacation in a couple of days, so there’s going to be a bit of a break in my journey of exploration. I hope to pick up the thread once more on my return and forge ahead. Let’s see why else I find worth writing about :-)

Socialize: del.icio.us | digg | Furl | reddit | Rojo | Spurl | Technorati | Yahoo My Web

Codephobia

Grady Booch had come to Manchester for a couple of days last month and I’d gone for his talk about "The promise, The limits, The beauty of Software". There were several interesting themes explored, but it was one comment made by him, almost in passing, which really struck a chord with me

He mentioned how he’s seen organisations which seem as if they’re afraid of code and that jibes with what I’ve noticed as well. An organisation or project or team which is completely tied up with process, to the extent that almost nothing can be done without reams of paper being produced, is usually a victim of codephobia. Before a single line can be written or modified, several Word documents will have to be either produced or updated, items in the project tracking system will have to be massaged, meetings will have to be called, recalcitrant team leads will have to cudgelled into submission and developers will have to be dragged to their seats – their nails leaving deep groves in the carpeting as they go.

Now mind you, this isn’t the usual screed by frustrated programmers against the horrors of ‘Process’. The ability to track, managed and control a project is vital to it’s success. However, smart, successful firms realise that all of these things are a means to an end. The customer wants a working application, not reams of design documents and lists of tracked defects. All of these are meant to enable you to write code better and faster.

This simple concept however, seems to be beyond the ability of many people. They act as if the various artifacts of process adherence were what the customer was paying them for and not the actual application. At heart, it’s all about being afraid of putting pen to paper - of producing the code - because that’s when the inadequacies will start to show and the incompetence will start to bubble to the surface, like marsh gas from a bog. In other cases, it’s the managers fear of the unknown. Most project managers are code-illiterate and so fear what they cannot understand or accurately track. They understand documents though and are happy to lose themselves in them. They’d love to have people code in Word if they could. Notice the popularity of code generators in such teams. It’s a dead give away :-)

In any successful team that I’ve seen, the focus is completely on the code. Everything is subservient to it and team is made up of and lead by competent programmers. Only a minimum amount of ‘process’ is tolerated and it’s all about knuckling down and churning out working code as soon as possible. Codephobic teams postpone code generation to the last moment and have to be dragged to their compilers kicking and screaming…

Which is kind of poetically just, because that is exactly what their customers end up doing when they see the end result.

Because the stakes are so small

"University politics are vicious precisely because the stakes are so small."
-- Henry Kissinger

I’m not a fan on Henry Kissinger by any means, but even the Devil can be right sometimes. It’s not university politics which concerns me though, but the sometimes byzantine intrigue which takes place within even relatively minor projects.

I’ve found office politics to be in turns, disgusting, depressing and distressing. It can occasionally be amusing, but only in the slightly tight-lipped, pathetic way an argument between two drunken bums might be amusing. What it usually is, is soul-destroyingly depressing. It sours the atmosphere of the entire project, splits teams and causes you to loose respect for people you’ll probably have to deal with on a daily basis.

And it’s usually so petty. The motivations are so base; jealousy, imagined slights, over-reaching ambition and profound insecurities. The tactics are so filthy; lies, rumour mongering and gossip. And what are the fruits? The imagined goals are usually minor things like a promotion, a slightly better appraisal or even something as petty as sucking up to management. The eventual result is usually bitterness, a loss of respect of others for ones-self and the debasement of your own soul.

How is one to deal with it when it starts? I have no clue. The people who are of this mould are usually incorrigibly bent out of true. You can try to straighten them out or bend yourself to fit their twisted psyche, but it’s usually futile. Both your resistance or compliance will trigger something within them which will make you their target. The base (and it is very base) cause, is that deep down, they actually enjoy the cut and thrust of it. And that won’t change until they do.

So should one get down in the mud and wrestle with them? Nope. You’ll feel disgusted with yourself, while they’ll be loving it. The best policy is to keep your interaction with such people to a minimum and just get on with doing a good job. They’ll either make so many enemies, they’ll finally be pushed out…

…or they’ll become the CEO.

In the eye of the beholder

What is art?

Is it only something like this or this? Or may be even this or this or this. How about this? Is this art or just the random scribblings of a demented monkey? Does splattering paint on a canvas qualify as art? If it does, then gulls are consummate artists. With better taste too.

Can you call this art? Or this, or even this. Is the man who shits in a jar and then sells it as high art, an artist? If poo in a bottle is art, then I've got Picasso beat. Every day.

Everything up until now has been the work of man (or ape). Is art something which necessarily has to be the creation of Man?

Is yes, then what about this? Or this or this or this? True, you need a computer to really bring these images to life, but they’re just mathematical functions. Order from chaos. Can we perhaps broaden our definition to include anything that’s beautiful to look at maybe? This looks great, but few would call the scene ‘art’. They’d call the photographer an artist though. But what about the programmer who wrote to code for the Mandelbrot program? No one calls him an artist even though both of them are simply capturing the beauty of nature. How does that make any sense?

Or does it? One can argue that given the same camera and scenery, I’d end up capturing disappointing images of mis-framed peaks, but given a computer and the algorithm, any one can produce the exact same images of chaotic functions. Maybe that human spark is key?

So if art isn't necessarily just something pretty (though it must be pleasing to the eye) and everything humans produce isn't necessarily art (though human involvement is necessary), then ‘art’ must lie at the intersection of these two planes.

So what is art?

Art is beauty, which is the result of skill or talent.

fin.