Sunday, September 20, 2009

Hill climbing problem with your career

Chris Dixon's has an interesting post on hill climbing as it relates to career development. In short, his basic premise is to imagine your career is a 3d surface and your goal is to get as high on this surface as possible. He then outlines a couple of algorithms to do this.

In explaining some trivial solutions I was struck by a common problem in people who work for a living. We are creatures of habit and we love the known. In addition, the known pulls at us due to it's immediacy, after all, it's right HERE, right NOW, I'm 100% certain of what it is. If I where to start venturing out into the unknown, I better have a reasonally high expectation of something positive happening. If I'm going into the unknown AND I'm fairly sure I'm going to have a negative outcome, it takes an inhuman (maybe vulcan) amount of logical reasoning to act counter to that intuition.

Put another way, I believe the reality of life is that this 3d surface metaphor is further complicated by the idea that it is more like an undulating surface whose peaks and valleys are constantly moving. This means that the job of moving upward is more active that explained in his post. In addition, this makes application of the metaphor make more sense from an economic perspective. Sure, being a big 3 executive is a sweet gig if you can get yourself into that position... but is it STILL that way?

People who think they deserve bailouts or stick around in dead industries or crappy jobs while bemoaning their state of affairs aren't necessarily stuck at a suboptimal peak. They could very well have hit a pinnacle at some point and their mental model of where they are was frozen while the reality shifted beneath them. They KNOW they are at the top of the highest mountain, why should they bother to look around and verify this information? In my mind, this should have the cartoonish consequence of them hovering in mid-air like wile-e coyote for a few moments before they realize they are sitting on nothing and then plummeting to the ground.

What does all this mean? It means if you're in a "sweet gig", but think it totally sucks and all you can do is complain about it, you're probably experiencing the mental stress of trying to reconcile the fact that you think you're at the top of the mountain, but in fact the substance that is holding you up is actually sheer mental stubbornness. If you cannot see this, you you may need to strike out into the unknown to reform your mental model of the world and begin to again see reality as it is.

What is also means is that if you think you're at the bottom of a deep dark valley, but things seem surprisingly OK, you might actually be in a better place than you really think you are (think a boat riding up a wave... it doesn't take a lot of effort). You might also do well to strike out and see what's around you.

Sunday, September 6, 2009

The Curse of Knowledge

I'm currently listening to an intro to psycology course lecture series and during a lecture on intelligence, the speaker mentioned "the curse of knowledge" (it's lecture 12 in this series ocw Brain and Cognitive Sciences. This is the problem caused by a difficulty in seeing the perspective of someone who DOESN'T know it.

This means that when I'm describing something to you, I inherently assume you know what I know. I have to consciously stop and think to try and figure out what you may or may not know and develop an idea about how to present what I know in a manner that you will be able to understand. It turns out that this is often difficult, especially if you have a LOT of knowledge about a specific subject. That is not to say it's impossible to see things from the ignorant party's perspective, but it requires a conscious effort to change perspective.

I see this all the time and it is particularly evident where I'm working right now. We have a fairly complicated set of software components that are used to do some fairly simple things (that is a whole other story). When speaking with the folks who built the fairly complicated things, many of their explanations are littered with assumptions about what needs to happen based on their inherent understanding about the inner complexities of what was built. As we get further away from the people who built the system, the assumptions begin to shift to people who have NO IDEA about the inner working, but have VERY GOOD ideas about what the software is supposed to do. These two camps are separated by a VERY wide chasm of knowledge that is NOT shared by both sides.

My current task is to get developers to divorce themselves from their current perspective and start looking at things from the perspective of their ultimate customers and how the software is valuable to them. Nobody outside our team really cares about the inner workings of our software, but we're so focused on these details that we've essentially misplaced our priorities. I feel this is causing friction because we are doing a WONDERFUL job of building software, but I'm convinced that we are building WONDERFUL software from the perspective of someone who wants to build wonderful software, not from the perspective of someone who wants to have a wonderful solution to a business problem.

Overall, the "curse of knowledge" has many implications outside of software development, here are some other links I found interesting on the topic.

Wednesday, September 2, 2009

Elusive software requirements

Those who have been following know that I moved to a new position about a month ago. I took over as a development lead for a project that was "just about ready" to go live. In speaking with the guys in charge, there was some concern because it seemed like some apparently simple changes where causing HUGE pushback from the development team. In addition, the application actually didn't even work by ANY measure... oh yeah, everybody could get a little tiny piece of it to work on their machine, but there's a whole other story about how we couldn't even actually build the project without having first built the project....

So I've spent an enormous amount of time (weeks) navigating the murky waters of "requirements" looking for the iceberg that caused this hole in our development boat. At first, I was told thing like "well, the requirements keep changing", and "well, things are kinda fluid, we just gotta be flexible". In addition, I had to listen to daily bitch sessions from the developers about things "always changing and nobody knows what the hell we're supposed to be doing".

Me being the FNG I jumped up and said "We're not doing ANYTHING that isn't spelled out in the requirements!" This had the effect of calming the development team down, but we where still seemed to have a problem of the software doing unexpected things. In addition, it was taking a god awful amount of time to do even the most seemingly simple thing.

So, I sat down with a BA, a manager (or two), and a couple of external consultants (sounds like the intro to a bad techhie joke) and started to talk about some of our problems... "what are we supposed to do here?" and "what about over here?". At this point I started getting a variety of answers... one person would say "just show a drop-down", another person said "oh, that's not really important", another person said "I wrote down EXACTLY what was supposed to happen about six months ago, it's spelled out in the requirements doc". About 5 times during this conversation, the director of IT's head turned beet red and I could actually hear the blood pounding in HIS ears.

This scenario played out repeatedly over the next couple of hours essentially ending with a VERY uncomfortable feeling on my part. The next morning a largish binder (80+pages) showed up on my desk with a manager + BA handing it to me saying "here are the requirements, make sure everyone understands them" (I'm paraphrasing, they really were much more polite).

Frankly, I was FUMING internally at this point. And then my inner a-hole came out and a fairly unfortunate conversation occurred. Luckily, at that point a number of largish prod support emergencies happened and I was thankfully whisked out of the conversation.

After calming down and driving around in my rental car for 45 minutes at lunch (don't ask, yet another story) I resolved to sit down and read all the requirements cover to cover.

My first discovery was that the requirements where VERY detailed, I mean, they where SUPER detailed. I instantly lapsed into my middle school coma of too much information. In addition, once I discovered things that somehow made sense to me, they seemed to be subtly different than my understanding of what was supposed to happen.

Worse yet, I discovered that about 70+% of the work we where doing was nowhere outlined in the requirements... I'm now trying to find the secret decoder ring that explain WHY we're doing all this work for seemingly simple requirements. The nonfunctional requirements were evidently stored in someone's head and nobody had accounted for them in any of the time-lines.

Well, the good news is that I've got plenty of opportunities to make things better, the bad news is that we blew our date by a huge factor... More to follow and I wade through the mists of requirementland.

We have a meeting about some new functionality tomorrow that ought to be REALLY entertaining.