Friday, December 23, 2005

Are there no truly original ideas left?

Look it what I found. Openomy.

Let's see now...

  1. Online File System? Check.
  2. Tagging? Check.
  3. REST based? Check.
  4. 1 GB space free? Check.
  5. Transparent Integration coming? Check.
See this blog entry for an introduction to Openomy.

Sounds real similiar to this and this, doesn't it? :-)

Thursday, December 22, 2005

4 Rules for the Practical Entrepreneur -- Fragmented Markets

Ian Landsman has written an excellent article for the practical entrepreneur in this article. Definitely worth the time spent reading it.

Thursday, December 15, 2005

SLoPIS (sloppies): Slow Loading Pages of Infinite Size

I've sometimes wondered how I'd instantiate and maintain a call-back link between a client and a server on the Internet. That is, I want the server to update the client as and when various events occur without the client having to specifically poll for them. This is very easy to do over an intranet, but seems almost impossible over the Internet because most clients will be behind restrictive firewalls and proxies. Using an arbitrary protocol over a randomly chosen port is not going to work. So is using polling the only way to go?

One way out of this conundrum might be to use something I've christened SLoPIS (pronounced: sloppies) - for Slow LOading Pages of Infinite Size - in a moment of whimsy.

What your client will do is issue a GET command to a particular URL, which will be generating the actual call-backs. The server starts to send back a 'page' with any queued events or a ping every 5 seconds to keep the connection alive. To any entity in the middle, the page simply appears to be very large and very slow loading, but not otherwise unusual in any way.

So lets say I have a REST based file system or something and I want to be informed of any file changes on the server. I can open up a connection to http://some.filesystem.com/SLoPIS/filechanges which is, lets say, a Java Servlet on the server. As part of the request I send my authentication data and the server responds with the any queued events. e.g.

<html>
<body>
DELETION: /mnt/some/file
...

If there's nothing to report for awhile and the connection is in danger of timing out, it'll send a ping, like so:

...
PING
...

to continue the downloading of the 'page'.

Now this isn't a true call-back, because the connection has to be initiated by the client, but for most applications I can think of, this is not really such a problem. To receive, the client just keeps an ear to the sloppy and reacts to any events sent over it.

So no more issues with firewalls or proxies and no need to create a special hole in them to get your application to work! :-) I can see this being very valuable for AJAX based web-apps.

There can be issues with load and running out of ports from a client source IP if there are a lot of NAT'ed users coming in from the same IP, so it's not a perfect solution. However, it may be more appropriate than polling in many situations.

On Ennui (on-we)

"This is the basic state of the creative soul when work has no meaning and brings suffering. In this state, employees feel they are doing the same things day after day. They repeat the same tasks, fill out the same forms and talk to the same people. They work in an environment they cannot change. They are merely the executors of other’s projects. They believe their bodies are the extension of other people´s minds.

In this stage, employees are not supposed to be creative. Don’t think, just do it! They are told. They don’t feel their work is important because they feel replaceable. Finally, they are not satisfied with what they do. When someone spends half their life doing something they don’t enjoy, it impacts their soul. Indeed, it has a deep impact.

This is the stage of Sysiphus. Sysiphus was a character of ancient Greek mythology. He represents two states of the soul at work: suffering and lack of meaning. Albert Camus explained the fate of Sysiphus with these words:

“The gods had condemned Sisyphus to ceaselessly rolling a rock to the top of a mountain, whence the stone would fall back of its own weight. They had thought with some reason that there is no more dreadful punishment than futile and hopeless labor”.

But the soul tends to move. It is like a river that always finds a channel."

-- The Life Cycle of the Creative Soul

Wednesday, December 14, 2005

A tag based file system (with Bayesian Auto-tagging)

So having been foiled by OmniDrive in my desire to create an Internet based virtual drive, I've moved on other, perhaps greener and defintely less populated pastures.

If you'll remember, I spoke about adding tagging support to the virtual drive I was talking about. I said that we could "... add tagging support to the virtual drive. You can tag files and folders and view virtual 'tag' folders with links to those files. Mainstream OS's don't have a tagging mechanism for files, so we'll have to add meta-data through file names. e.g. end file names with a special character and the tags (i.e. myfile.txt#work,proposal,text) which will be stripped off before being saved to the virtual drive."

I won't be working on the 'Internet' part of the virtual drive, but I can certainly implement this idea. Why not create a FUSE based file system with tag based virtual folders? Use either folder names (/mnt/tagfs/here:are:some tags/myfile.txt) or file names (/home/user/myfile.txt:here:are:some tags) to add tags and then use virtual tag directories to navigate through those tags? You can use mv to change the tags associated with a file etc.

In addition, we can have a Bayesian Auto-tagger which learns which tags you've previously used for files of a certain type and then automatically tags them appropriately if no tags are supplied. The more you tag, the better the auto-tagger.

Who knows, I might decide to run with this one! :-)

The saga continues...

Relevant links:

Monday, December 12, 2005

Another one bites the dust

No sooner do I start thinking about a globally accessible, scaleable, encrypted virtual drive that someone announces their intention of releasing just such a product! :-(

Check out OmniDrive. Pretty much what I was aiming for. They've even got their eyes set on Google and have a very interesting blog entry on The Economics of Online Storage.

Well, back to the drawing board! :-)

Friday, December 09, 2005

The Fine Art of Programming

A fine and growing collection of online programming guides and such. A link worth saving and revisiting.

Thursday, December 08, 2005

A virtual drive in every pot

I've been mulling over the idea of a virtual drive on the Internet for some years now. Witness my ham-handed efforts at http://zfs.sourceforge.net for an early example. Well, I recently resurrected the idea of writing something like it now and it's interesting to see how my thoughts have evolved.

ZFS as I initially envisioned it was to be a network of automatically replicating file servers and the use case in my mind was a university file server. There would be a mapping of many users to a single (virtual) server, with the system (internally a cluster) having to scale to handle as close to an infinite number of users as possible.

Lately, I've been thinking more along the lines of writing a 'Net Drive' type application. A virtual disk I can mount from any machine connected to the Internet and treat as a local drive. Companies like xDrive and iDrive and MangoSoft already offer something of the sort. However, their offerings are targeted more towards the business user. I personally feel there's a massive untapped market of casual users who might be interested.

Imagine having 1GB of space available to you online and directly accessible via a virtual drive. Directly save documents, media files etc. to the virtual drive and access it from anywhere through another computer with the same drive mounted in or through a web interface. Share your password with several people and have them save in the same drive if you wish, give them a URL to the data on your drive or just share certain folders. Boom, you've just eliminated the need for a hard disk on our PC. Internet appliances, here we come!

This type of application would be ideal for someone like Google to create and I can see them stepping into this field sometime soon. It's a classic Google app. You need to scale almost infinitely, but that's easy because you can create slices of the virtual resource and limit the number of users accessing each slice. Want to support more users? Add more slices.

Take Gmail as an example. It's probably got hundreds of millions of users, but unlike the University use case, the users have a many to many (or from another perspective, a one to one) relationship with the system. That is, unlike university students, gmail users are not interested in checking other people's mail or accessing a common email account, or even sharing their email account. This makes it much easier to slice up the virtual space, assigning a limited number of users to each slice and scaling the slices. So Gmail is probably made up of thousands of individual computers, each supporting let's say 1000 users, fronted by an authentication cluster. When a user wants to log in, he goes to gmail.com, is authenticated and then redirected to the individual machine he shares with 999 other people. If you want to add another 1000 users, plug in another machine. You can keep scaling horizontally till infinity for all practical purposes. The authentication datastore will eventually become a bottle-neck, but you can support a enormous number of users before you hit that wall. *

If Google were making a virtual drive, they'd do something similiar. As new users signed up, they'd be assigned to different machines, upto a certain max cap. Just like gmail, users have a one to one relationship with their account. That is, they're only interested in the contents of their accounts and have no need to access anyone elses account or a common store. This makes it trivial to scale exponentially.

Now this is a great product to make and market, except for one small problem; there are already a whole bunch of people out there doing the same thing. So we need to differentiate ourselves from the pack.

One way to do that is to add tagging support to the virtual drive. You can tag files and folders and view virtual 'tag' folders with links to those files. Mainstream OS's don't have a tagging mechanism for files, so we'll have to add meta-data through file names. e.g. end file names with a special character and the tags (i.e. myfile.txt#work,proposal,text) which will be stripped off before being saved to the virtual drive. Users can also publicly 'share' tags.

Other features we can offer are:

  1. Fast file indexing and searching and maybe even mapping/linking files to each other based on content etc.
  2. Clients for hand-helds with disconnected operational ability
  3. Single-click integration with Flikr, Del.icio.us etc.
  4. Rsync based transfers
How are you going to pay for all this? Advertising. Have the virtual drive folder show text/banner ads and the website as well. Have premium accounts and dedicated machines for business users.

Who knows, I might work on this idea... or maybe not.

* Correction: It's possible to avoid turning the authentication store into a bottle-neck. One way to do this would be to have the store for a particular set of users reside on the machine assigned to them. So when you want to access the virtual drive abc.virtualdrive.com, you go to that URL and send in your username and password. If the authentication process running on that machine can't find the user, the login attempt fails.

This just leaves the DNS server as the bottleneck now :-)

Wednesday, December 07, 2005

PETA - People Eating Tasty Animals

If God didn't want us eating animals, why did He make them out of meat? :-D

A couple of friends and I were discussing the various methods used to slaughter animals (over lunch, when else) and we got to talking about the Halaal method (where a stroke through the neck severs every vein, artery and the wind-pipe, but leaves the spinal column intact) versus the Jhatka method (lit. jerk - where the animal is decapitated in one stroke). The debate revolved around which method caused the least pain to the animal, with most people automatically assuming that the Jhatka method was more painless.

I disagree.

In both methods, the animal eventually becomes unconscious due to a lack of blood going to the brain and the resultant drop in blood pressure. In the Jhatka method, since the animal is decapitated and the head is no longer attached to the body, we don't see the animal kick about and grunt as animals being slaughtered are wont to do. However, the sensation of 'pain' is interpreted by the brain and as long as that is active, the animal will suffer. The head being separated from the body doesn't make any difference.

Since in both methods, the blood flow is disrupted and in one we have the additional pain of the spine being cut through, the Jhatka method should logically be the more painful of the two, with the added disadvantage of not keeping the heart going for as long as possible to clear out as much of the pathogen carrying blood as possible.

It's just an illusion that Jhatka is more merciful, brought on by the stillness of the decapitated animal corpse. A final verdict awaits the time when it will be possible for us to measure and quantify 'pain' as a value...

Hungry kya? :-P

Tuesday, December 06, 2005

SQL Injection and XSS Attacks

Some topics everyone involved in web development must read at least once:

  1. SQL Injection Attacks by Example - It's a lot easier than you think. And yes, you customers will try it out some of the standard approaches out of idle curiosity if nothing else.
  2. Real World XSS - You'll be surprised at the sites which are vulnerable to attacks of this nature.
  3. More XSS
  4. And still more XSS

Mark Cuban - Success & Motivation

Blog Maverick - The Mark Cuban Blog

In Essence:

  1. Be driven
  2. Know the industry you're in, inside out.
  3. Use 1. and 2. to ensure you're ready when Lady Luck strikes.
  4. You only need to be lucky once...

Cuban comes across as an intense, driven, workaholic; kind of like my current (successfully entrepreneurial) employer :-D. It's a bit depressing to think that the only way to free yourself from the chains of a 9-5 life, is to handcuff yourself to a 00:00-23:59 one!

Friday, November 18, 2005

ColorMatch Redux

Thanks to this, I can now tell which shirt will go with which trouser.

Breakthrough!

Tuesday, November 15, 2005

Slacker Manager: Full frontal personal marketing assault

I'm always interested in reading about innovative interviewing techniques and this post from the Slacker Manager is really interesting.

An innovative way to stand out from the crowd without using cheap gimmicks or tricks. Nice.

Corporate Cargo Cultism

"In the South Seas there is a cargo cult of people. During the war they saw airplanes with lots of good materials, and they want the same thing to happen now. So they've arranged to make things like runways, to put fires along the sides of the runways, to make a wooden hut for a man to sit in, with two wooden pieces on his head to headphones and bars of bamboo sticking out like antennas--he's the controller--and they wait for the airplanes to land. They're doing everything right. The form is perfect. It looks exactly the way it looked before. But it doesn't work. No airplanes land." -- Richard Feynman in "Cargo Cult Science"


Acting like a large corporation will not make you one. The trappings of such companies - the meetings, the excessive documentation, the blind insistence of following process and damn the consequences - are a by-product of becoming successful and large, not their cause. It can in fact be argued that many large companies are actually diseased, paralysed by their size, their culture of CYA and their unwieldy processes.

No wonder then that mega-corps that still retain their drive and ambition are looking to inject some much needed small company vibe into their moribund veins. They want the agility, the passion and the personal touch that only a smaller company can provide.

So if you're a smallish company, don't blindly adopt the practices of the big boys simply because that's the way they do things. They do things the way they do because they have no choice. It's difficult to maintain the personal touch when you have a team of 1000 under you, and you need the reams of documentation when both the product and the team size is massive.

Companies like Apple and especially Google know how to retain the common touch. While the press releases of Microsoft look like they've been written by Borg drones, Googles communiques read like they've been written by the actual people behind the product. People who are genuinely excited and happy about releasing something they've worked on and sweated over and believe in. Real people.

Free flowing information and passion is what makes small, committed start-ups such great places to be. Don't stifle the life-breath of your organisation by trying to be excessively professional and all grown up.

You can make your little firm look like a flake off of a blue chip, but it's not going to make the airplanes land...

Monday, November 14, 2005

GTD TiddlyWiki

This has got to be the flat-out, most mind-blowing thing I've seen all month!

A fully functional wiki in a single, standalone page of HTML. Using the magic of Javascript and CSS, the authors have managed to make a complete application which requires nothing more than a modern browser to operate. Just save the empty template to your own computer, double click it and away you go!

It's got regex incremental searching, tagging, snazzy animations... the works. It's replaced my text only todo file habit of several years and that is quite a milestone. It's got some basic keyboard shortcuts which make editing your entries (or tiddlers) quite easy. Since it's HTML, you have the ability to create links, add formatting etc. which isn't possible with a plain ol' text file natch.

It's got all the advantages of a text file (free form, completely portable, searchability) plus a whole lot more. Absolutely fantastic! :-)

It's derived from TiddlyWiki so you can use the original instead of the fork if you want.

Monday, November 07, 2005

The 3 Urinal Salute

Pass by the urinals in the office loo and *gush*, *gush*, *gush*, you just might be greeted by a volley of cheerful flushing. The urinals spot the presence of what they think is a satisfied user and smugly activate their self cleaning mechanisms...

Nothing like a 3 urinal salute to make you feel appreciated; and a little moist.

Introducing the Non-intuitive Manager

In this series of articles, I want to explore some of the practices we as managers follow because instinctively, we think they're effective. Unfortunately, a little experience and thought quickly shows that many of these practices are actually counter-productive and occasionally, down right destructive.

People spending far too much time chatting on the intranet? Log every key press and word sent across! Getting the feeling that people are wandering the halls with far too much abandon? Restrict access to the various parts of the office! Want an update? Drop in unannounced and demand it on the spot!

The basic problem is that of a lack of empathy. We forget - usually because it's been a while since we've been in a similar situation - how we'd feel if placed in the same spot as the one we've just stuck someone in. At other times, we lose sight of the fact that not everyone comes from a similar background as us, and so an incident which we may shrug off as being insignificant may actually be taken quite seriously by others.

As straight-forward as the problem is, it's solution is simpler still. Just about every philosopher, religious figure and ascetic has expounded some version of the Golden Rule; Do unto others as you would have done unto you. In other words, place yourself in the situation of your hapless team members. Would you like to have someone breathing over your shoulder, eyeing your every move? Or tracking every keystroke of your messaging activity? Probably not. And if something like this would make you uncomfortable, chances are, it makes other people uncomfortable as well.

There is a danger here though. Not everyone is the same. As I mentioned earlier, people come from varied backgrounds and have varied expectations and tolerances. For example, behaviour that you might consider intolerable micro-management might be comforting to someone who's less sure of himself and wants a little hand-holding. You have to make allowances for things of this nature and that's where experience helps, I suppose...

Anyway, that's why I decided to name the series, "The Non-intuitive Manager" since good management is sometimes about going against what your gut tells you is the right thing to do. You can't always rely on your intuition or your instincts to guide you, since these are often completely off the mark. If managing was something that came naturally to most people, there wouldn't be so many horror stories about the "supervisor from hell". The only way to be sure about what you're doing is to test things out and then measure the effects.

The only way to be sure that what you're doing adds to the teams well-being and overall happiness is to implement your policy and then see if it's had the desired effect. If so, continue to refine it, taking constant feedback from your victims team members as you go. If not, take suggestions and feedback and try something else. You have to implement a constant loop of innovation, testing and feedback if you want to be sure that what you're doing is having the desired results. Relying on whims and fancies and ruling by decree is for tin-pot dictators...

It isn't the Non-intuitive manager you've got to be worried about, since he understands the dangers of relying on instinct. It's the natural, intuitive ones who are completely confident about their inherent talents who are terrifying!



Articles in the series

  1. Introducing the Non-intuitive Manager
  2. The Non-intuitive Manager [How doing what seems 'right' can be counter-productive]
  3. The Non-intuitive Manager: The Semantic Weight & Half-Life of Words

Thursday, October 20, 2005

Resume Writing 101

So my cousin is thinking of hopping employers and sat down to update his resume a while ago. While going over his efforts, I was struck by a rather obvious 'insight'; a resume is just a marketing document!

I can only blame my complete stupidity for not having thought of this before. All this time, I've always thought of the resume to be, well, a curriculum vitae. A summary of one's education, professional history, and job qualifications... as Answers.com puts it. A numbered list of enraged ex-employers, dubious educational institutions and dodgy projects to be paraded before incredulous HR drones. A stack of paper, with as dense a mass of black on white as possible. Something to win your next assignment with by boring - or more satisfying still - bludgeoning, the opposition to death.

However if you really think about it, a resume is really the first step is marketing that most desirable of commodities.

You.

Like any good marketing campaign you've got to not only provide accurate information, but also dress it up in a manner that attracts and excites. Let's face it, there's not much difference between a good resume and what's on a packet of chips. You've got the ingredients, the promotional puffery, the endorsements by trusted spokespersons... What we've got to learn is what makes people buy one brand and not another. Especially since at heart, all they're really getting is some sliced, salted spuds and a warm feeling that's just impending indigestion.

It's - to a large part - all in the presentation.

So apply standard marketing techniques to your resume. Identify your target audience. Satisfy HR enough to get by their buzzword filter. Stand out enough so you're noticed by first level decision makers. Give HR their alphabet soup of acronyms and certifications, but make sure the tone is such that it also catches the attention of your (you hope) new superiors.

Write your resume taking into account the kind of company and industry you're aiming for. Be edgy and provocative if you're targeting a hipper crowd. Stay a little more conservative if you want to be a banker or accountant. Remember though, it's better to be specifically rejected than simply ignored. If you make something that some people hate, then others will love it. Design to elicit emotional responses. It's better to be hated than to not to make an impression at all.

Think of using colours, textured paper, blurbs from past employers/superiors, innovative designs... whatever. The people reviewing your resume plow through hundreds of boring CVs a day written by earnest hopefuls. You have to make an impact if you're going to get an invite to an interview...

...which is a whole 'nother packet of chips.

Monday, October 03, 2005

The Non-intuitive Manager: The Semantic Weight & Half-Life of Words

[You might wish to read the Introduction to the Series before you proceed...]

Words matter. A lot.

How you phrase something makes a big difference, especially when you're talking to people who are technically your subordinates. Now, this may seem like the most obvious statement in the world, but it's surprising how often it's forgotten.

What we tend to forget is that when the boss speaks, his words are mined quite intensely for subtleties and hidden meanings by his subordinates. A simple "Hello" might send some of your more insecure underlings into a panicky tizzy. Now this might seem unfair to you, but that's just the way things are. Human nature isn't slated to change anytime soon, so you might as well get used to it, bugs and all.

The problem is a basic one of power. The bossman has it, the wage-slave doesn't. The managers whims directly impact the average workers life in measurable ways, while things don't quite work vice-versa. It isn't surprising then that the communication flowing from the top is analysed much more obsessively than you expected it to be.

In other words (HA!) we need to examine...

The semantic weight and half-life of words


Description: You words and opinions, even ones joking expressed, are examined much more deeply for nuances than you'd like. Often, your subordinates build-up their stances or expectations on utterances which were hasty, or meant to be taken lightly. What seemed like banter to you, was taken as a reprimand; or your flippant remark about a company product has made people treat it lightly.

The Problem: People are apprehensive in nature. Real or perceived power resides in you in this relationship, not them. In addition, you're looked at to provide leadership and direction. People will emulate your stances and align their positions to the ones they think you hold. In addition, there's no expiry date on words, so something you said 6 months ago may still be shaping the teams perceptions about itself, the product or the company, long after the situation has changed.

A Possible Solution: Being aware of the issue is halfway to solving it. You have to be very careful with what you say. Your heart should be behind your tongue, as it's sometimes put, not before it. There's no need to be fake, or a bland, emotionless robot. Joke and josh around with your team with abandon, but remember the limits. Don't put anyone down, even in jest and never denigrate an in-house product or another team.

Having to be so careful may seem like an over-reaction.

But words matter. A lot.

Wednesday, September 28, 2005

The Non-intuitive Manager [How doing what seems 'right' can be counter-productive]

[You might wish to read the Introduction to the Series before you proceed...]

Managing people is easy.

Managing people effectively is not.

If managing people were easy and intuitive, Dilbert wouldn't exist. The fact that a cartoon strip about a disaffected white collar wage-slave not only exists but is widely read by the cubicle classes, is telling. Everyone knows or has worked under at least one PHB or been bombed by a wandering Seagull Manager, even other PHBs and SMs!

Of course, no one sets out to become a bad manager, so how come the corporate world is chock full of them?

The problem is fairly insidious. It seems to be that when people think they're doing the right thing, they're usually not. Our intuition about what is constructive behaviour is very often wrong.

A classic example is what I'll label the gratuitous walkabout:


The Gratuitous Walkabout


Description: This involves the manager in question sauntering up to his unsuspecting target on padded feet and ambushing him with a cheery "So, how're things going?". The surprised subordinate turns around and tries desperately to collect his thoughts, mumbling senselessly in the mean time. By the time he's managed to context-switch successfully, he's already succeeded in making a fool of himself and the manager has moved on in search of his next kill, smiling ruefully and shaking his head. Bonus points if he manages to squeeze in a heavy I-really-expected-better-from-you sigh.

The Problem: It's difficult for most people to instantly switch from problem solving mode to reporting mode, especially when they're tense and on the back foot from being surprised. Someone working hard at his job will find it difficult to context-switch and swap in the information required to answer your questions accurately and fully. Nervousness from being under the microscope doesn't help.

A Possible Solution: Instead of pouncing unannounced on your subordinates, it might be better to set up a regularly scheduled meeting to which they can arrive ready and prepared. Even giving someone a 5 minute warning before you go to meet them should be enough, if all you're looking for is a daily update. If you must insist on surprising them, then make some small talk for a few minutes to give them time to ready themselves and only then get down to business.