Sunday, December 27, 2009

Ira Glass on Storytelling

Ira Glass (of "This American Life" fame) has a 4 part VLOG on storytelling. I really enjoy listening to Ira when I have the opportunity and he has had some compelling stories.

My personal revelation is that I should look more to people I respect and study them. As someone who never formally studied creative writing or storytelling, it was interesting that the mechanics of building a story outlined by Ira were more natural and easy to follow than the myriad of rules found in various English composition books I've read. I believe this the difference between trying to define a simple set of rules on how to master the mechanics of doing something as opposed to learning to bend your mend into the shape of a master of the art.

A further revelation is that I realize that I am very often reading books about subjects written by people who aren't necessarily skillful or even successful at that particular subject. As an example, when reading about some very successful writers (written by the writers themselves) their techniques were almost always different from the techniques outlined in many books about creative writing.

In my own professional life, many technical books about programming or software development are written by folks whose technicals and skills I do not particularly admire. While this can be good because it gives a different perspective on things, I need to to remember to temper the amount of credence I give their potentially crappy ideas. Too often I spend time trying to do things the "right way" as told by a person with meaningless credentials, and a "better way" is staring me in the face and used effectively by practitioners of the art.

While it should be obvious, I'm now going to devote more time to seek out masters with contributions that I believe are cut above the rest to seek out and learn from them.

Monday, December 21, 2009

The elves leave middle earth

I just stumbled across a great article about the difficult transition from a startup into a more sustainable business.

The elves leave middle earth

I'm currently working in a place that is going through this change in the IT organization. Right now, we have a mixed team of heros and good performers.

A key problem I'm having is that one of the managers is having a tremendously difficult time even fathoming the difference between these two modes. As a former hero coder, he spends his time fiddling around with code and inventing new "cool" requirements and almost zero time even thinking about how to keep the team performing.

This has the predictable side effect of the team faltering and thrashing while this person sits around and bemoans how slow and confused the team is. Where I'm going is that if you're going to decide to change modes, you need to make sure you have a management team that is able to function in that mode. If it's necessary to move folks in/out of management to make things a good fit, don't be afraid to do so... waiting just causes MORE problems.

Saturday, November 28, 2009

ATI drivers on Karmic Koala

I set out to upgrade my older laptop to ubuntu 9.10 and I knew was going to have problems. Historically, the video card in this thing was a royal pain in the butt and just never seemed to work quite right.

I loaded the OS, logged in, and was rewarded with a jumpy/twitchy user experience. I started researching and found a thread about some tweaks a few folks made to make things better.

It turns out this didn't work for me, but a minor tweak to the tweak fixed me up.

Notably, I downloaded the radeonhd driver from synaptic, changed the xorg.conf (created above) to use radeonhd instead of radeon, then removed the bit that seems to have been autodetected by X11 for thier hardware.

This seems to have fixed my problem (for now). In addition, I can run the compiz visual effects (yeah expo).

Here is my config:


Section "ServerLayout"
Identifier "X.org Configured"
Screen 0 "Screen0" 0 0
InputDevice "Mouse0" "CorePointer"
InputDevice "Keyboard0" "CoreKeyboard"
EndSection

Section "Module"
Load "dri2"
Load "dri"
Load "dbe"
Load "extmod"
Load "glx"
EndSection

Section "InputDevice"
Identifier "Keyboard0"
Driver "kbd"
EndSection

Section "InputDevice"
Identifier "Mouse0"
Driver "mouse"
Option "Protocol" "auto"
Option "Device" "/dev/input/mice"
Option "ZAxisMapping" "4 5 6 7"
EndSection

Section "Monitor"
Identifier "Monitor0"
EndSection

Section "Device"
Option "EnableDepthMoves" "True"
Option "EnablePageFlip" "True"
Option "DMAForXv" "True"
Option "AccelDFS" "True"
Option "ColorTiling" "True"
Option "RenderAccel" "True"
Option "VGAAccess" "True"
Option "AccelMethod" "EXA"
Option "DRI" "True"
Option "MigrationHeuristics" "greedy"
Option "TripleBuffer" "True"
Option "EXAOptimizeMigration" "true"
Option "EXANoComposite" "No"
Option "BackingStore" "true"
Option "AGPMode" "8"
Identifier "Card0"
Driver "radeonhd"
EndSection

Section "Screen"
Identifier "Screen0"
Device "Card0"
Monitor "Monitor0"
Option "Accel" "True"
SubSection "Display"
Viewport 0 0
Depth 24
EndSubSection
EndSection

Section "Extensions"
Option "RENDER" "Enable"
Option "Composite" "True"
Option "XVideo" "Enable"
Option "XINERAMA" "False"
EndSection

Section "DRI"
Mode 0666
EndSection

Sunday, November 22, 2009

Bad code gone wild

I've been writing software for quite some time (first computer) and have seen a lot of strange stuff.One thing I realized is that there are fundamentally two types of bad programmers.

#1 bad programmers that gets better.
#2 bad programmer that don't.

I can tell you I'm a #1: I routinely do things that are not optimal and "could have been done a better way", but don't beat myself up about. It is what it is and as long as the code "gets the job done", I'm OK with that. In addition, I'm OK with developers writing occasionally bad (wrong/suboptimal) stuff because I believe most folks can learn and get better. To this end, I found this amusing Bad Code Offsets

But occasionally I run across a #2: There are people who do things in broken and difficult manners and no amount of exposure to "better ways of doing things" will yield fruit. These folks, once they find "a way", it becomes magically "the only way" and will likely never find a "better" way. They will copy-paste write entire applications from snippets found on the internet. They will write functions that do the OPPOSITE of the the function says. They will do things that make you stop and ponder if they are deliberately trying to do the wrong thing (industrial espionage?).

For my own example, I can thing of a developer many years ago who was writing an asp application that was connected to a database with millions of rows. Their initial decision was to use a client side cursor and iterate through the resultset filtering rows that way. It worked fine on their dev box with 100 rows of data, but once they started testing, (with millions of rows) it seemed to just "hang" and never come back.

I overheard a pretty heated discussion between the developer and a DBA about why this was a bad idea, but the DBA seemed to not be getting through. I chimed in and tried to help out by drawing pictures but nothing seemed to help. To make matters worse this "developer" then complained to management that they where not getting "any help" in resolving the "performance problems with the test database".

From this developer's perspective the problem was really caused by the DBA team not setting up the test database server properly. It had nothing at all to do with this person's code because it worked on their machine.

For more fun stories, read The Daily WTF
One of my personal faves THE ULTIMATE STATE SELECTOR

Wednesday, November 18, 2009

Ubuntu Cafe

I just read some news about If microsoft, Linux, and OSX were cafe's

I'm wondering... would people pay a few bucks to hang out with some linux (Ubuntu?) folks to help them fix their windows boxes so they run well? I know a lot of people who don't care which OS their running as long as things "work" and don't break.

The latest few ubuntu releases sure seem like good candidates, I wonder if you just charged a cover charge to "non technical folks" and used that money to pay the rent as well as diet coke for the "technical folks" if you could make a business out of this. They techno weenies can run around being the gurus and helping folks out and the other folks can get super customized machines. Could even maybe even set up virtual windows machines on them so quicken and other non-wine software works.

Interesting idea, not sure I have the business chops to make it happen. I know we've been running linux here for almost 2 years and the only thing that we miss are #1 video games (use consoles and flash games) and #2 itunes (virtualbox).

Wednesday, November 11, 2009

Crazy autogenerated email signatures

I was just rereading a few posts from many moons ago (yeah, like 6 years) and realized that my employer at the time was postpending a crazy addendum to every one of my email messages. I really wonder how many people read this thing and exactly how legally applicable it is since the messages are now archived all over the dang internet.

As an example from 2003:
http://www.servlets.com/archive/servlet/ReadMsg?msgId=411933&listName=taglibs-user

Perhaps this thread from a few months before that caused the company to start applying the extra stuff to the message:
http://mail-archives.apache.org/mod_mbox/struts-user/200312.mbox/%3CC29830220D2FA74BA88CB5C62C946B32C0AF03@uskihsvpex05.kih.kmart.com%3E

!Doh!

Sometimes you need to break the rules

In honor of veteran's day I thought I post a "war story" to illustrate point.

As a buck sergeant in the US Army many moons ago (1995), I was deployed to Hungary in support of IFOR to enforce the military portion of the Dayton agreement. At some point, I was forward deployed with a team mate to go to all of the personnel service detachments and set up their digital communication links back to germany (and subsequently the US).

While standing around on the tarmac in preparation to get on a plane heading to Tuzla, a few of the other soldiers grabbed their ammo and packed it into the middle of their 'A' bag. I thought nothing of it, but the guy I was with walked over and asked what they where doing. It turns out the Air Force did not want us flying with ammo and they wanted us to give their ammo to them. These other soldiers (old school combat arms guys) where NOT about to give their ammo to some tech sergeant who may or may not remember to give it back, so they hid it.

Once we where getting ready to be manifested onto the plane, the loadmaster came around and asked everyone to give him their ammo. I dutifully did so watching with disgust all these other dishonest folks just shrug and say "I don't have any".

My self-rightousness was very short lived. As soon as we landed, I sought out my ammunition, only to realize it had been left in Hungary. Worse yet, after calling around trying to find it, it turns out my ammo had been turned into the ammmo supply point as "lost". Now I had two problems: #1 I was driving around in a war torn country with NO ammunition and #2 It had been reported that I LOST ammunition (probably worse from some folk's perspective).

Even more interesting, as I drove around in various convoys, I kept trying to figure out how to get some of my own ammunition. Because my unit was actually stationed in Hungary, nobody was willing to give me any ammo because they had signed for it and didn't want to get in trouble for "losing" it.

If I was smart, I should have asked the combat arms guys I ignored in Hungary how to get some ammo instead of trying to work within the convoluted alternate reality of "the system".

In summary, rules are important, but sometimes they're also stupid, counterproductive, and are waiting to be broken. I understand that the Air Force doesn't want unwashed army personnel shooting holes in their planes, but in retrospect, the potential downside of carrying my ammo and lying was a lot better than the downside of giving up my ammo. I'm not even clear on why they thought I shouldn't have my ammo, I guess they thought maybe someone could have gone crazy and tried to hijack the plane?

Monday, November 9, 2009

dd-wrt on a linksys router.

I have a couple of old linksys wrt54g routers lying around. They stopped working 100% over the years and I just bought another one instead of trying to fix the one I had.

Surfing the internet I stumbled across a linux firmware image for them that allowed you to reconfigure them to do some wicked cool stuff http://www.dd-wrt.com/site/index. Since I just donated an old laptop (without a wireless adapter) to my son and I've been wanting a way to get my old ps2 hooked up the the wireless router, this looked like just the trick.

I downloaded the firmware images, flashed the router, and suddenly I had a little linux router I could configure and tweak to my heart's content. I could do really cool stuff like connect wirelessly from my router to another wireless router and bridge the networks together. This means my ps2 upstairs can now connect to my old linksys and it will forward all the traffic to my new wireless access point downstairs (sweet!). It also allowed me to bump up the radio power as well as super tweak the radio (like I can tell the right antenna to transmit and the left one to receive).

If you're a hacker and have some spare time or a need to create a mesh of wireless access points, I HIGHLY recommend this package.

three thumbs up.

Sunday, November 8, 2009

Building a custom debian package

We run debian (or ubuntu) as our platform of choice, and largely this is great. If we want software we just apt-get install the package and we're ready to go. The downside is, however, that occasionally a package won't exist (or only an older one will exist). In my case I wanted tomcat 6 on lenny. Previously what we would do is each server install would require a set of instructions to follow (untar foo into bar, symlink this or that, etc). This led to a situation where we had 4 different ways our various server where set up.

In order to make this "better", I wanted to build a .deb to make the install/removal/upgrade process easier. I could have done the "right" thing and build the proper package from the source, but it turns out it is relatively simple to build your own .deb binary, so I did this instead.

I used these instructions
http://tldp.org/HOWTO/html_single/Debian-Binary-Package-Building-HOWTO/

After about 4 hours (there were some false starts), I had a package that could be installed and removed. Hopefully this will go a long way to making our server installs go more smoothly.

Next step will be to add the extra scripting pieces to allow us to configure our tomcat instances instead of hacking xml files.

opennms install on ubuntu

We currently user nagios for network monitoring, but it has a buttload of crazy config files to even remotely work correctly. As an alternative I decided to give opennms a try to see if there wan easier way to keep track of thing. Note, my home network is probably more complicated that many small businesses as I've got around 10-12 computers/printers/wireless bridges/media servers/etc running at any given time and I have android, linux, windows, Mac all connected at various times.

The reasons for trying opennms where:
#1 free (as in both speech and beer)
#2 it purportedly supports autodiscovery (please work!)
#3 it's java based (no shell scripts??)

So I installed via synaptic, and promptly dicovered that it requires postgres out of the box. I additionally had a couple of "gotchas", which I'll document here.

Off the bat, I tried to start via sudo /etc/init.d/opnenms start and it immediately complained and said I needed to do some extra setup steps. Let me say that there ARE instructions here for those weak kneed sissys who actually NEED them. They are, however, not tailored for ubuntu and "the Debian way".

In the context of the above directions (yes Nancy, you'll need to read them) on ubuntu, OPENNMS_HOME is /usr/share/opennms.

You'll need to get postgres and sun-java6-jdk. For some reason I had the openjdk installed and it complained that this MIGHT not work, so I installed the sun jdk and reconfigured to be the default:

sudo update-java-alternatives -s java-6-sun

then

sudo /usr/share/opennms/bin/runjava -s

I then tried this

sudo /usr/share/opennms/bin/install -dis

But got exceptions connecting to the database. Evidently, by default, the postgres user cannot connect via a password. To get around this, I created a new account "opennmsadmin" that is a superuser that can connect via a password.

sudo su - postgres
createuser -s postgresadmin
createlang plpgsql opennms
psql
> create user opennms password 'opennms';
> create database opennms;
> create database opennmsadmin;
> alter user opennmsadmin with password 'opennmsadmin';
> grant all privileges on database opennms to user opennms;
> grant all privileges on database opennmsadmin to user opennms;

\q
exit

I realize that there are a couple of extra steps in there... I probably could have just done a
createuser -s postgresadmin -P (to set the password) and skipped the alter user step.

Anyway...

I then could edited the connection info contained in /etc/opennms/opennms-datasources.xml

sudo vi /etc/opennms/opennms-datasources.xml

and then run the setup.

sudo /usr/share/opennms/bin/install -dis

It SAID it completed correctly, so now I can run

/etc/init.d/opennms start

and wait while the JVM consumes all available resources on your machine (just kidding, but it IS slow to startup IMHO).

after that, you can navigate to:
http://localhost:8980/opennms

You can then configure which ranges of IPs should be scanned (there are also instructions on how to do this a different way here):
http://www.opennms.org/wiki/Quick_Start

Seems to work, autodiscovery is NICE... need to talk to the network guys tomorrow.

Friday, October 23, 2009

Requirements are not necessarily required

In the grand tradition of being difficult, I'm rediscovering an antipattern that has killed projects in the past. I'll call it "too specific requirements".

The symptoms are stacks and reams of requirements that you MUST implement, even if there are better ways to solve the apparent problem. Case in point: We have a rock solid requirement to build a user preference management screen. Unfortunately, most of the preferences we need to collect are (IMHO) better collected and maintained in situ.

As an example, one requirement is to "enable the user to select their default payment method". Of the hundreds of possible solutions, the requirement implies that we should have a special screen where users can find this option, select it, then never see it again when checking out their items.

This sounds great on paper, but I'm struck by the notion that a user would EVER know or think to go to some other screen (preferences? My Profile? not sure what to call it) to save their default payment type. In addition, the extra problem is now that we've hidden their payment type, how will they know how to select a new payment type?

A better way (unproven as yet) might be to collect this option WHILE they are checking out and selecting it. This way, instead of conducting extensive user training for our "stupid users", we can make it more natural.

Sunday, October 4, 2009

Think like a genius

Questia has an interesting article about "how to think like a genius" http://www.questianewsletter.com/newsletter/volume-5-issue-5/index.htm?CRID=nullCRnull&OFFID=newsletter20091004l#bigidea . I'd have to say this is spot-on and a good

Adding my own thoughts to the above, I've noticed that I rarely do new or innovative things by "trying", I'm more innovative when exploring interesting side effects. I would go further to say that lucky people are simply the ones who can creatively extract value from more situations than other people. I often hear people sharing their misfortunes with me and quite often they have a negative spin on things that happened to me that I thought where golden opportunities, but they simply saw as obstacles.

Which reminds me of a story...

A gentleman farmer at the turn of the century was being overrun by rabbits and went into town looking for an expert marksman to get rid of the pests. On his way, he passed a farm with dozens of bull's eyes painted on a fence and in the the exact center of each one was a bullet hole. The farmer was amazed and went up to the house to find this remarkable marksman. On approaching the owner he asked if the owner had fired all those rounds. When the owner responded in the positive, he asked: "You've got to be the best shot ever, how many years of practice did it take to become so good?" The markman replied "Actually I only started shooting today. After I got done shooting at the fence, I just went and painted circles around the holes"

Genius!

Saturday, October 3, 2009

The hazards of microbenchmarks

I recently had lunch with my team and had the dubious fortune of sitting across the table from Mr Knowitall. As we recently installed SONAR, he mentioned his surprise at our "critical performance" issues, namely that we're using a + b to concatenate strings instead of StringBuilder in a few dozen places. I dismissed this as "not really a critical problem", which spawned a heated discussion about the merits of using StringBuilder over normal string concatenation in Java.

He ended up issuing a direct challenge that Stringbuilder is 25% faster than string concatenation. He also stated that I could build a test harness and see for myself and bet that if he was right I would buy lunch...

Aside from kicking myself for getting into a ridiculously pointless argument that I've had am million times before (he also tried to drag me into the "you better check .isdebugenabled() before calling .debug() in log4j because otherwise you're taking a performance hit, but I resisted this one).

I tried to get a word in edgewise about how I had this discussion with a guy from Sun (Brian Goetz) who REALLY knows this crap a LOT better than me he basically said it was a fool's errand to try and rely on microbenchmarks for real performance indicators as the JIT compiler, garbage collector, and a million other things will make your benchmark crap. There was no reconciling Mr. Knowitall though, he was absolutely certain he was right and determined to make sure I was beat into submission about my wrongness.

So after dropping my daugther off at cross country, I created this class:

package javaapplication1;

import java.util.Random;

public class Main {

public String getValue() {
return "AAAAABBBBBcccccdddddEEEEEfffff";
}
Random r = new Random(6);
private String getRandom() {
String a = String.valueOf(r.nextInt() * 100900);
return a.substring(0,4);
}

public static void main(String[] args) {
Main one = new Main();
Main two = new Main();
Main three = new Main();
Main four = new Main();
Main five = new Main();
Main six = new Main();
int count = 10;
for (int i = 1; i < 1000; i++) {
two.stringBuilder(count);
two.finish();
one.stringaddition(count);
one.finish();
three.stringBuilderMethod(count);
three.finish();
four.stringadditionMethod(count);
four.finish();
six.stringadditionMethodRandom(count);
six.finish();
five.stringBuilderMethodRandom(count);
five.finish();

}
System.out.println(one.time());
System.out.println(two.time());
System.out.println(three.time());
System.out.println(four.time());
System.out.println(five.time());
System.out.println(six.time());


}
public String time() {
return name + ": " + duration;
}
private String name;
private long start;
private void start(String name) {
this.name = name;
this.start = System.currentTimeMillis();
}
public long duration = 0;
private void finish() {
duration += (System.currentTimeMillis() - start);
}
public String stringBuilder(int count) {
start("stringBuilder");
StringBuilder sb = new StringBuilder();
for (long i =0; i < count; i++) {
sb.append("AA");
}
return sb.toString();
}

public String stringaddition(int count) {
start("stringaddition");
String output = "";
for (long i =0; i < count; i++) {
output += "AA";
}
return output;
}

public String stringadditionMethod(int count) {
start("stringadditionMethod");
String output = "";
for (long i =0; i < count; i++) {
output += getValue();
}
return output;
}

public String stringBuilderMethod(int count) {
start("stringBuilderMethod");
StringBuilder sb = new StringBuilder();
for (long i =0; i < count; i++) {
sb.append(getValue());
}
return sb.toString();

}

public String stringadditionMethodRandom(int count) {
start("stringadditionMethodRandom");

String output = "";
for (long i =0; i < count; i++) {
output += getRandom();
}
return output;
}

public String stringBuilderMethodRandom(int count) {
start("stringBuilderMethodRandom");
StringBuilder sb = new StringBuilder();
for (long i =0; i < count; i++) {
sb.append(getRandom());
}
return sb.toString();

}

}

I put the random stuff in there just for fun to see variations with the overhead of "normal" stuff one might do.

Here are the results:
stringaddition: 6
stringBuilder: 6
stringBuilderMethod: 5
stringadditionMethod: 22
stringBuilderMethodRandom: 15
stringadditionMethodRandom: 11
BUILD SUCCESSFUL (total time: 0 seconds)

Woa! first off, in the trivial case, there was no real difference... In the method call, string concatenation was 4x slower and in the "do some work" version, stringbuilder was SLOWER.

Case closed, I win!!! Of course, if I cared to investigate further I would discover something... like, when I ran it a second time, I got these results...

run:
stringaddition: 6
stringBuilder: 1
stringBuilderMethod: 10
stringadditionMethod: 24
stringBuilderMethodRandom: 7
stringadditionMethodRandom: 12
BUILD SUCCESSFUL (total time: 0 seconds)

Huh... Now it appears that StringBuilder is a CLEAR winner, 6x faster in the best case, a still a good 20% in the worst. The really bad news (I think) is that our margin for error is now much larger than our potential savings from these sort of refactorings.

Well, what now? It turns out, by fiddling with the JVM settings I can get crazily different results that make one or the other of these look a little better or worse. Also, if I change the JIT settings and/or call the methods in a different order, I can make it look like StringBuilder is consistently slower.

After cracking open the source for the runtime library, the implementation of string + string actually USES StringBuilder, the only real difference is that every call MUST call toString() so there is additional garbage generated. As I was trying to say at my (now mostly wasted) conversation with this guy was: "it's more complicated than simply answering which method is faster".

The lessons to learn from this are:

#1 Don't prematurely optimize. The fact is, the code that this person was worried about would be called about 1 or two times per day. This means changing the code would have a net gain in performance of a couple of milliseconds per year... Hardly a stunning victory for performance. I had to call it a hundredthousand times to get any reasonable gain in performance.

#2 Don't try to reason with unreasonable people. Some folks just REALLY want to be right, they don't really want to have a conversation. When they start bullying and pounding their chests, figure out a way to get out of the conversation without engaging in a pointless debate... (but don't necessarily back down as this will often encourage the boorish behavior).

Sunday, September 20, 2009

Hill climbing problem with your career

Chris Dixon's has an interesting post on hill climbing as it relates to career development. In short, his basic premise is to imagine your career is a 3d surface and your goal is to get as high on this surface as possible. He then outlines a couple of algorithms to do this.

In explaining some trivial solutions I was struck by a common problem in people who work for a living. We are creatures of habit and we love the known. In addition, the known pulls at us due to it's immediacy, after all, it's right HERE, right NOW, I'm 100% certain of what it is. If I where to start venturing out into the unknown, I better have a reasonally high expectation of something positive happening. If I'm going into the unknown AND I'm fairly sure I'm going to have a negative outcome, it takes an inhuman (maybe vulcan) amount of logical reasoning to act counter to that intuition.

Put another way, I believe the reality of life is that this 3d surface metaphor is further complicated by the idea that it is more like an undulating surface whose peaks and valleys are constantly moving. This means that the job of moving upward is more active that explained in his post. In addition, this makes application of the metaphor make more sense from an economic perspective. Sure, being a big 3 executive is a sweet gig if you can get yourself into that position... but is it STILL that way?

People who think they deserve bailouts or stick around in dead industries or crappy jobs while bemoaning their state of affairs aren't necessarily stuck at a suboptimal peak. They could very well have hit a pinnacle at some point and their mental model of where they are was frozen while the reality shifted beneath them. They KNOW they are at the top of the highest mountain, why should they bother to look around and verify this information? In my mind, this should have the cartoonish consequence of them hovering in mid-air like wile-e coyote for a few moments before they realize they are sitting on nothing and then plummeting to the ground.

What does all this mean? It means if you're in a "sweet gig", but think it totally sucks and all you can do is complain about it, you're probably experiencing the mental stress of trying to reconcile the fact that you think you're at the top of the mountain, but in fact the substance that is holding you up is actually sheer mental stubbornness. If you cannot see this, you you may need to strike out into the unknown to reform your mental model of the world and begin to again see reality as it is.

What is also means is that if you think you're at the bottom of a deep dark valley, but things seem surprisingly OK, you might actually be in a better place than you really think you are (think a boat riding up a wave... it doesn't take a lot of effort). You might also do well to strike out and see what's around you.

Sunday, September 6, 2009

The Curse of Knowledge

I'm currently listening to an intro to psycology course lecture series and during a lecture on intelligence, the speaker mentioned "the curse of knowledge" (it's lecture 12 in this series ocw Brain and Cognitive Sciences. This is the problem caused by a difficulty in seeing the perspective of someone who DOESN'T know it.

This means that when I'm describing something to you, I inherently assume you know what I know. I have to consciously stop and think to try and figure out what you may or may not know and develop an idea about how to present what I know in a manner that you will be able to understand. It turns out that this is often difficult, especially if you have a LOT of knowledge about a specific subject. That is not to say it's impossible to see things from the ignorant party's perspective, but it requires a conscious effort to change perspective.

I see this all the time and it is particularly evident where I'm working right now. We have a fairly complicated set of software components that are used to do some fairly simple things (that is a whole other story). When speaking with the folks who built the fairly complicated things, many of their explanations are littered with assumptions about what needs to happen based on their inherent understanding about the inner complexities of what was built. As we get further away from the people who built the system, the assumptions begin to shift to people who have NO IDEA about the inner working, but have VERY GOOD ideas about what the software is supposed to do. These two camps are separated by a VERY wide chasm of knowledge that is NOT shared by both sides.

My current task is to get developers to divorce themselves from their current perspective and start looking at things from the perspective of their ultimate customers and how the software is valuable to them. Nobody outside our team really cares about the inner workings of our software, but we're so focused on these details that we've essentially misplaced our priorities. I feel this is causing friction because we are doing a WONDERFUL job of building software, but I'm convinced that we are building WONDERFUL software from the perspective of someone who wants to build wonderful software, not from the perspective of someone who wants to have a wonderful solution to a business problem.

Overall, the "curse of knowledge" has many implications outside of software development, here are some other links I found interesting on the topic.
http://37signals.com/svn/posts/213-the-curse-of-knowledge

Wednesday, September 2, 2009

Elusive software requirements

Those who have been following know that I moved to a new position about a month ago. I took over as a development lead for a project that was "just about ready" to go live. In speaking with the guys in charge, there was some concern because it seemed like some apparently simple changes where causing HUGE pushback from the development team. In addition, the application actually didn't even work by ANY measure... oh yeah, everybody could get a little tiny piece of it to work on their machine, but there's a whole other story about how we couldn't even actually build the project without having first built the project....

So I've spent an enormous amount of time (weeks) navigating the murky waters of "requirements" looking for the iceberg that caused this hole in our development boat. At first, I was told thing like "well, the requirements keep changing", and "well, things are kinda fluid, we just gotta be flexible". In addition, I had to listen to daily bitch sessions from the developers about things "always changing and nobody knows what the hell we're supposed to be doing".

Me being the FNG http://en.wiktionary.org/wiki/FNG I jumped up and said "We're not doing ANYTHING that isn't spelled out in the requirements!" This had the effect of calming the development team down, but we where still seemed to have a problem of the software doing unexpected things. In addition, it was taking a god awful amount of time to do even the most seemingly simple thing.

So, I sat down with a BA, a manager (or two), and a couple of external consultants (sounds like the intro to a bad techhie joke) and started to talk about some of our problems... "what are we supposed to do here?" and "what about over here?". At this point I started getting a variety of answers... one person would say "just show a drop-down", another person said "oh, that's not really important", another person said "I wrote down EXACTLY what was supposed to happen about six months ago, it's spelled out in the requirements doc". About 5 times during this conversation, the director of IT's head turned beet red and I could actually hear the blood pounding in HIS ears.

This scenario played out repeatedly over the next couple of hours essentially ending with a VERY uncomfortable feeling on my part. The next morning a largish binder (80+pages) showed up on my desk with a manager + BA handing it to me saying "here are the requirements, make sure everyone understands them" (I'm paraphrasing, they really were much more polite).

Frankly, I was FUMING internally at this point. And then my inner a-hole came out and a fairly unfortunate conversation occurred. Luckily, at that point a number of largish prod support emergencies happened and I was thankfully whisked out of the conversation.

After calming down and driving around in my rental car for 45 minutes at lunch (don't ask, yet another story) I resolved to sit down and read all the requirements cover to cover.

My first discovery was that the requirements where VERY detailed, I mean, they where SUPER detailed. I instantly lapsed into my middle school coma of too much information. In addition, once I discovered things that somehow made sense to me, they seemed to be subtly different than my understanding of what was supposed to happen.

Worse yet, I discovered that about 70+% of the work we where doing was nowhere outlined in the requirements... I'm now trying to find the secret decoder ring that explain WHY we're doing all this work for seemingly simple requirements. The nonfunctional requirements were evidently stored in someone's head and nobody had accounted for them in any of the time-lines.

Well, the good news is that I've got plenty of opportunities to make things better, the bad news is that we blew our date by a huge factor... More to follow and I wade through the mists of requirementland.

We have a meeting about some new functionality tomorrow that ought to be REALLY entertaining.

Wednesday, August 26, 2009

Upgrading Linux ATI drivers on ubuntu

OK, normally I don't do tech stuff in my blog, but this has been a long standing frustration that, now that it's fixed, I feel the need to share.

Background: A year or so ago I bought a fairly inexpensive laptop from Tiger direct. It was a toshiba satellite A215-S740 and only cost around $600. This is a fairly decent machine and I intended to use it for work. It came preinstalled with Vista, but I typically run Linux (ubuntu to be exact).

I thought I'd run vista for a while and see how I liked it, tired after two days of the constant interrogation. I felt like Vista was an overly agressive Gestapo agent... so I installed Ubuntu (ahhh, much better).

My first problem, however was that the built in ATI driver did not support 3d. At the time I was playing with building 3d visualizations of our systems and I REALLY needed 3d (because software 3d was way too slow). I found the ati proprietary drivers, installed them, and everything seemed OK acceptable, but not quite Great.

Some examples of problems where:
Desktop effects didn't work
Changing resolutions and hotplugging monitors didn't quite work.

Admittedly these where minor so i basically ignored them. Every month or two I would download the latest ati drivers and then attempt to upgrade. Invariably I would waste 2 hours fooling around with them and finally give up.

Recently, I took off the gloves. I had some vacation time and decided I was going to solve this problem once and for all. After much wrangling, I discovered some things that really seemed necessary to get things to work.

#0 This I kept forgetting and I think had something to do with it
sudo rmmod fglrx
sudo rmmod ati_agp

#1 Uninstall the old drivers.
yeah, really, just do it. If you want my advice, hit on startup, pick the #2 option (safe mode or whatever it is) and then pick "go to root shell". Some alternatives might be to switch to another console, log in, and stop GDM, but I'm trying to illustrate the most "foolproof"

For those that don't know how, the steps I took to uninstall where:
sudo dpkg -r fglrx-amdcccle
sudo dpkg -r fglrx-modaliases
sudo dpkg -r xorg-driver-fglrx

#2 Create the ubuntu .deb files from the installer.
sudo ./ati-driver-installer-9-3*.run --buildpkg

#3 install the ubuntu .deb files
sudo dpkg -i fglrx-kernel-source_8.593-0ubuntu1_i386.deb
sudo dpkg -i fglrx-modaliases_8.593-0ubuntu1_i386.deb
sudo dpkg -i xorg-driver-fglrx_8.593-0ubuntu1_i386.deb
sudo dpkg -i fglrx-amdcccle_8.593-0ubuntu1_i386.deb

#4 reconfigure the driver
sudo aticonfig --initial --resolution=0,1280x800 -f

#5 restart!

Some key points...

#1 I used a BUNCH of other techniques, but I never uninstalled the drivers first, nor did I stop the kernel modules (#0 and #1) I think these where critical.
#2 When running aticonfig, use the -f option to write the changes into the xorg.conf file. I kept getting kernel panics when I didn't include the -f.

Monday, August 17, 2009

Transition from two guys in a garage to multinational company

There is an interesting difference between how folks do things when its "two guys in a garage" and "a really big multinational company". The first thing I would say is that you can run a company of either size the same way. The second thing I would say is that you CANNOT run a company of either size the same way EFFECTIVELY.

If you are a working in a web based software startup with 16 people and you have 2 developers, 4 project managers, 6 vice presidents, a CIO, CTO, CFO, and CEO... you are destined for failure (if you don't know why, I bet your title had the word "Chief" in it). On the other hand, you have an established company with 1600 employees, 24 well known brands, a B2B application that does a billion dollars a year in sales, you might need a couple of managers who try to keep things organized. You really don't need 40 developers all "doing their own thing" with little rhyme nor reason.

I've been on both ends of this over the years as well as a number of places in the middle and it's interesting to watch people try to blindly apply patterns that worked in one environment in a radically different one and be puzzled when they don't seem to work.

Folks... there is not golden solution to every problem. If there was, somebody would have patented it by now and be grand imperial commander of the universe. The fact is, every new and unique thing we do creates and demands new patterns and solutions because they create new and unique problems that have not been completely solved. In addition, even the simple passage of time can cause a solution that worked great 5 years ago to be remarkably silly now even if it's in a remarkably similar context as "the last time".

Saturday, August 8, 2009

Cakes, Onions, and Ogres have layers, are you sure your code needs them?

Anybody write java code out there?
Anybody have a data access layer?
How about a business layer?
How about a service layer?

Why?

I've heard all manner of crazy answers like:
It makes my code more flexible.
I can dynamically swap out components for one layer or another.
I need to keep things isolated so that a problem in one layer don't impact the other.

Convince me, because I'm just not sold at this point. I went through a number of projects with this fad back in the 2001-2005 timeframe and none of the theoretical benefits never seemed to emerge...

Project insanity

I've been rereading my journal (not blog) and realized that every year, for the last 3 years, I've been engaged in a crazy "meet the date at all costs" project that ultimately ended up being canceled around the March to July timeframe. This year was no exception and I raised red flags, drew charts, rang warning bells, blew whistles, rang gongs, frankly I think I did everything but set myself of fire and for some reason... we continued to mindlessly tool along using an outright crazy set of expensive tools to "deliver" a project that was totally behind schedule, didn't do what we wanted, and frankly was one of the most ridiculously avoidable failures I've had the misfortune to be involved with. Mind you I'm sure it ended up "working" and probably wouldn't be canceled this year because it had good buy-in from senior management, it's just we did it the hard and expensive way and relied on heroics of individual people instead of doing it the smart and economical way.

For a postmortom, somehow it seemed like I was speaking a different language when I tried to discuss the problems and how we might avoid or fix them. Worse yet, it seems like I developed a reputation among my management team as a difficult and inconsolable whiner who just didn't want to buckle down and do what he's told. Oddly enough, our CIO had urged me to make sure I communicate things, but I felt a personal sense of loyalty to my management team to not expose problems to their bosses. In retrospect, I think that loyalty was misguided and while it's unfortunate, I think it probably will cause the problem to persist for quite some time.

In any event, I personally think I utterly and abjectly failed in my role as a technical leader on that project. I stopped trying to resist letting the train run off the tracks and arrived at the conclusion that I did not posses the requisite communication skills to explain the problems and how they should have been solved... Frankly, my solution was to shut up and leave the company.

I've since moved to another company... More to follow

--Mike

Monday, August 3, 2009

I met me today

The strangest thing happened today...

I met myself at lunch.

Or rather, I met someone who sounded a lot like me. You see, I just started a new job and I went out to lunch with few of my new team members. Part of the lunch digressed into a bit of a bitch session about stupid crap at work. It was kinda nice to have folks be so upfront and just vent instead of dodging around the issue and playing little word games to try and see if I would finally understand what they really meant.

Someone just said "I think so and so is a dumbass" and that was that. No "well, you know (make up little story that is hopefully a metaphor I would understand) and then proceed to allude to this fact over the next 30 minutes, it was just flat out and out "so and so is a dumbass".

I like it... I'm not saying that this person was right or wrong, it's my first day, how could I know? But what I like is that I don't need to guess what that person thinks. While I supposed they could be gaming me or have some ulterior motive, it certainly seems like they just wanted to get their position on the record. That gives them +1 in my book, though a high score is in the millions, so it's not a terribly impressive feat.

Thursday, July 23, 2009

What is development time?

I'm responding to a recent email chain where I made the statement "this will only take 8 hours of development time" and stopped. It occurred to me that we had already spent 20+ man hours of meetings to decide if we should do this particular thing, how we should do it, and who should do it. If we had simply said "do it" we would have saved 18 hours labor... or even better, could have developed 3 alternative solutions and spent a couple hours deciding which one was best.

It's interesting though because in the chain of meetings, emails, and other administrative things, when I sat down and looked, there where only two people who actually had "work" to do. I define "work" is defined as the act of transforming an idea into a useful piece of software. Everyone else was simply there to "help" get the work done.

When I start looking back at my problems estimating... it isn't really that developers and technical people are bad at estimating, it's that they're bad at anticipating the number of ancillary things that will crop up from a relatively simple solution. If I estimate 8 hours of effort (especially when I REALLY think it should only take 2, but I'm padding a little because I know the person who will make the change won't be quite as fast as me), I fully anticipate it to end up taking 20-30 hours of extra labor for various people who will never actually DO anything, but want to sit down and talk about it.

So now my question is: Is my true estimate 2 hours? 8 hours, 30hours, or something else?

Tuesday, July 21, 2009

Broken Design

I've noticed an interested effect on corporate portals or extranets. They are often set up so that external access is allowed, but only certain applications/links actually work. This leads to a situation where you either #1 always VPN in (often redundant and useless) or #2 randomly have links break that will not work for you until you VPN in.

A better solution might be to make the links visually different if they only work internally and perhaps even put a little broken symbol next to it. This would be best if it only showed up when you are navigating to the site from outside.

Tuesday, July 7, 2009

Difference between strategy and tactics

What is the difference between a strategy and a tactic? As far as I can tell, the commonly accepted difference is scope. A strategy is typically cross functional and wide and a tactic is smaller and narrower with higher detail. Anyone else have better ideas?

Tuesday, June 30, 2009

Leadership styles

As a person who ends up being a de-facto leader on any team I'm on, I often am stuck trying to subtly nudge the "real" leaders into thinking about the act of leadership. I have the advantage in this regard in that I've participated in hundreds of hours of leadership training over the years and have had a lot of experience working with teams to get things going. A quick primer that I think is very instructive can be found here.

So if you're a leader and your team is doing CRAZY things and they aren't happy and you aren't happy and you aren't getting the results for the organization that you require....

My first statement is "yes it's YOUR fault"
My second statement is read this again
My third statement is: adjust your leadership style to fit the situation. Do NOT treat 30 year old software developers like children, because they will begin to act like them. This should be no surprise.

In addition, make sure you attend any leadership training made available to you, it certainly cannot hurt... Certainly if your organisation is making leadership training available to you free of charge, it would be a really good idea to attend.

Sharpening the saw

I follow coding horror and recent read an interesting piece titled "sharpening the saw". This resonated with me and I felt compelled to expound on the concept a little.

In brief, sharpening the saw refers to the notion that working harder with a dull saw is not as effective as sharpening the saw, then cutting the wood. Obviously this is a generalization, but it alludes to a problem I see all the time and I honestly like the metaphor.

Leaders in software development organizations need to build an understanding about the importance of keeping their developers on the cutting edge. A developer using vi or notepad to write software is wasting quite a few brain cells on things that a modern IDE can do. In addition, a developer who only uses COBOL and has done so for 20 years is having their brain turned into a solid rock that will eventually be unable to produce anything helpful (note to COBOL programmers, this applies to java, fortran, ANY development language/tool)

I know, I know, there are a number of alpha geeks out there who code everything in vi, as a matter of fact, there are probably a few folks who actually write bytecode by hand. They will also all say "and I'm still faster than (name crappy programmer they've compared themself too)". In addition, someone will probably also send a link to a web server/3d rendering too written COBOL. What they're missing is that they're not as fast or effective as they could be.

The important point, however, is that a technology leader needs to help build an understanding among all the players about the importance keeping the saw sharp. Programmers know what they are doing and what the state of the art is. They also need to be given the best tools that are economically feasible so that they aren't wasting energy because of the tools they've been given.

If the excuse you use is "there's no TIME to do any of that" then you either #1 aren't fit to be a tech leader or #2 your leader is crushing your team. Either way I would encourage you to take personal responsibility for keeping your saw sharp.

Some specific examples of dullness:

#1 Do you have web developers with 13" monitors that flicker ? - Compared to the annual salary of even an entry level software developer, the cost of a 22-24" widescreen is trivial. Having workmen's comp claims from blind developers with migranes is going to be a fruitful legal area in a few years....mark my words.

#2 Do your developers go to the same tool for EVERY solution (java, C#, COBOL, whatever)? This is closely related to the Golden Hammer problem, but is really a symptom of developers not being exposed to new ways of doing things.

#3 Do you routinely hear things like "that's impossible, because {insert some problem that is highly technical, but irrelevant}"?

Monday, June 29, 2009

The ulitmate software development tool

I find it amazing that I work for a company that sells thousands of different hand tools, but we still seem to thing there is "one ultimate software development tool" and waste time trying to force professional developers to use a single tool to develop software.

Why is this? What is it about developing software that makes people think you really only need one tool to do it? Why is it obvious to everyone EXCEPT the software developers that Rational Application Developer is the "ultimate tool" for the job and is all anyone would ever need?

I'm to the point where I want to suggest a solution used by many auto repair shops.... Give all the developers an annual stipend to purchase the tools they think they want to use and be done with trying to figure out and enforce the "standard" tool.

Thursday, June 25, 2009

The journey is the destination

The other day someone was asking me a bunch of questions and they where getting exasperated because they didn't know ANYTHING about what I was talking about.
They asked something like: What's JSON? HTTP? Grails? Bytecode? JVM? How do I know how to find out what I need to know? How do YOU know all this stuff? Did you document any of this? What class should I take?

I responded with a link to http://google.com and http://www.wikipedia.org (which, BTW was not well received).

The fact of the matter is, I know a lot of people who think they really want to know things, but they have poor learning skills. They are of the mindset that they can learn something and then be done learning it. In today's world more than ever this is really just not practical.

For example, if you knew everything there was to know about web development two years ago and stopped there, by most measures you are pretty out of date (I'd say you were a dinosaur, but I'm an extremest). The rate at which new things to know are being generated is momentous, and you cannot hope to learn and be "done" because your knowledge is getting outdated faster than you can acquire it.

The key is that you need to constantly be learning... every day, every thing you do should be different and you should learn something from it. If you just do the same thing over and over because you've acquired some proficiency with it, you should push yourself to try new and innovative techniques that will teach you something along the way. If you keep doing the same old thing, you will fall hopelessly behind and never be able to catch up.

This poses interesting problems for, if the body of knowledge is growing at an exponential rate (it's certainly greater than linear), when are you "finished" with your education? What's the point of a 2 week class on a particular software package (or a 4 year degree in comp sci) if you only can ever do or know what you where exposed to during that time? If that is the case, you will never get better and be hopelessly out of date almost immediately (even with a PhD or postdoc credentials).

Learning isn't really a destination, it's a journey. It is a never ending journey, so accept this, and stop thinking you know everything because you learned it last year... you're out of date, hit the books, hit the internet, talk to your peers...

Sunday, June 21, 2009

Lazy Man's web hosting

I recently took over as the "IT Guy" for a local organization I volunteer for. As a guy who spends a lot of time working on web development, I seemed to be a good fit for the job. The website was basically a bunch of static files with a little bit of javascripty rollovers thrown in for good measure.

My mandate was to try and bring the site into the 21st century with things like: blogging, email notification, user forums, group calendar, and other features that would make things more interactive. At first I figured I'd just write some stuff myself, but then I started looking around. Twitter (microblog + sms notification), Google (calendar, online word processing), and Facebook (just about everything else) provide a bunch of these things for free. When put together, these give a lot of functionality for little more than the time to set up a few accounts and link the various pages together.

Now the site is MUCH more interactive, and we actually have no real "code". I sense a place for this sort of expertise in the market that I'm going to start exploring. Maybe I'll call myself a "social networking application professional" (SNAP).

Friday, June 19, 2009

Cultural regression

I work on a development team that for the last year or so was making a lot of progress toward being effective at #1 delivering useful software, but #2 responding to stimulus with the appropriate level of response.

At this point, however, we've fully regressed to our previous state of overreacting to very small stimuli. A small configuration change to a server and we act as if we've discovered the Russians (no offense to anyone from there) have launched their entire nuclear arsenal at our office building. Any small hiccup results in finger pointing, conference calls, and multiple teams running off to the excuse factory to build something to explain why something went wrong.

Heaven help the poor sap who tries to actually fix or avoid the problem, he will be beaten down with a litany of things that SOUND really bad and scary, but in fact have little or no real impact. I believe this is happening because folks are fearful of their jobs and want to give the impression that they are hard workers who are dedicated to getting things done. It seems that in this case, there is little interest in solving the real problem, because reacting the the problem of the day comes with reward for "lot's of hard work" and of course appreciation for the "countless nights and weekends by dedicated associates".

The cynical side of me thinks that many people are over-blowing the real problems in order to make sure everyone understands how IMPORTANT they are. One perennial problem that impacts me personally is an age old ( in our organisation) debate about source code control. For one, I am stunned by, and somewhat ashamed of, the sheer number of hours we have wasted on this debate.

At one point we gave up and just used Clearcase, but the sheer effort required to keep this tool afloat caused us to go underground and begin using SVN again. We then ended up formalising SVN within the organisation, but this ironically has caused MORE problems. More problems stem from the fact that the change management team, because they don't understand or WANT to understand the developers, just tells people to "do whatever they want" with SVN instead of having them follow good practices. This has the predictable effect of causing inexperienced developers to fall into obvious and predictable traps that could easily have been avoided had the change management team spent some time helping craft good practices using the tool.

Right now, we're busy attributing problems in the development process to SVN, RAD, Eclipse, Groovy, and a million other tools. The problem is, as far as I can see, all of the problems are really being caused by PEOPLE not tools. The tools are just convenient excuses and witless pawns in the game of "it's not MY fault".

Friday, June 5, 2009

Agile Enterprise Architecture

I've been reading with great interest many things about enterprise architecture. It seems that in our organisation, traditionally at least, there is little thought about the structure of the business as it relates to information management. While it appears there has been great thought in restructuring the organisation around strategic objectives, we seem to always forget that the information needed to obtain these objectives is critically important and the technology teams to manage that information should be structured and aligned in the same or similar way.

This is really interesting to me because it seems to reflect a profound ignorance of the value that a technology organisation can provide.

http://www.agilearchitect.org/agile/index.asp
http://www.agilearchitect.org/agile/principles.htm

Wednesday, June 3, 2009

Top Down Management

Simply doing what the boss says even though you know it's stupid is irresponsible. I know many people who do a lot of stupid stuff at work and am constantly amazed at how often completely competent and intelligent people will do insane things just because the boss said so. It probably shouldn't surprise me, after all, history is replete with examples where a charismatic leader inspired millions of people to do really, REALLY bad things. It just stuns me that, in a corporate environment, people will wander off and do really stupid things without even letting the boss know it's stupid.

I think this is, in part, a problem in technology organizations because it is not necessarily a meritocracy, but the the people "in the trenches" like to think that it is. After all, how inspired are you going to be if, after 23 years on the job, you're at the top technical position in the company, and the are still 5 layers of managers between you and the top.

It's as if we are still trying to pretend that technology jobs are the same as factory jobs: You've got "management" who are college educated folks, and the "workers" who simple do what their told. Somehow this culture, in large organizations, still persists and causes no end to chaos.

We really need to come to grips with reality and realise that technology is a skilled trade and the masters of that trade and the ones who should be in charge. How quickly would the medical community disintegrate if the "head of surgery" was a communications major who never once performed and actual surgery. What would happen if the person in charge of plumbing for a huge skyscraper was actually an English major. Obviously, if these folks are skilled leaders they could be successful, but not nearly as much as if they where also masters in the trade.

Tuesday, June 2, 2009

Right to privacy

First off, I am not a lawyer (IANAL) so this is not legal advice and hell, it might even be wrong.
I recently got involved (read opened my big mouth) in a discussion about RFID in the context of providing extra security for individual financial transactions. A point was made that the security of this scheme is suspect because, after all, they make wallets that block RFID scanners. My response was that I wasn't sure this was a real threat to security or privacy, or a perceived one. I was attempting to illustrate these "RFID proof" wallets could likely be a gimmick by some huckster playing up the fears of the populace.

The response I got quoted the fourth amendment and I promptly got a little confused. I'm always a little uneasy when people mistakenly think the constitution grants them some explicit right to privacy. From my limited perspective, the point of the constitution is to limit the powers of government over the people, not to dictate what an individual can or can't do.

I would argue that it is probably unconstitutional for the government to outfit patrol cars with RFID scanners and then patrol the neighbourhood for stolen merchandise (in your house) or to set up checkpoints along the highway to search for contraband...

But is it is certainly not unconstitutional for a private business to engage in activities to determine what customers do while in their place of business. After all, it's private property, there is nothing compelling you to be there... I didn't sent my thugs to your house the grab you and force to to buy things in my store (website). You voluntarily left your home, drove to my store and gave up a great many rights to privacy by doing so.

All of this is ancillary to my original point which is... who is outfitted to track RFID tags to that level of detail right now? I recently did the math and a 64 bit tag has enough unique combinations to store 10 million unique codes for every human who has ever ever lived... EVER. I'm sorry, but I know for a fact we have a VERY hard time tracking and infinitesimally smaller number of things through our supply chain, tracking that number of unique tags is going to be a lot of work (not impossible).

Monday, June 1, 2009

The Human Factor

Jeff Atwood in Coding Horror mentions in his blog that:

The key word here is "fun".

When you interact with other people online ..

* sending an email to a mailing list
* posting on a discussion forum
* chatting on IRC
* revising a Wiki entry
* entering a blog comment

.. like it or not, you're participating in the world's largest MMORPG. Lurking is always free. Those that choose to go beyond lurking, to add some tiny bit of content to the web, do it because they find it enjoyable. On some level, they're having fun

Enjoyability of a user experience, even for a back office application is important. More than that, most people do what they do because at some level they enjoy it. If you structure your applications internally so that "doing the best thing for the business" increases the level of enjoyment at work, you're on the road to success.

Obviously, this can be taken to an extreme, but let's take a recent example. We recently deployed and evangelized yammer at work and a great many people have taken to microblogging as a communication tool. I'm not sure exactly WHY we decided to start this, but if the intent was to begin to break down barriers and get functional stovepipes to begin communicating better, somebody deserves a handshake, pat on the back, and perhaps even a beer at the local drinking establishment.

I won't go much further on extolling the virtues of using a microblog at work, but I noticed an interesting side-effect. People are using this tool, embracing it, and... shock of shocks! talking about non-work related subjects! From my perspective, this is GREAT, they've not only begun to openly communicate, but they're comfortable enough to begin to connect intellectually and emotionally with people they would have otherwise never even met (we've got 250,000+ employees).

On the other hand, folks in management can now see what people are talking about. This makes some folks very nervous, to the point that they refuse to use this tool. In fact, they approach me (one who perhaps spends TOO much time yammering) and say things, in hushed tones, like "do you know that the executives can SEE what you're writing?" "People might think you don't have any work to do and you'll get fired."

I won't go into my tirade about how water cooler conversations are as important as formal meetings at this point. I would, however, love to walk through the cafeteria, gym, coffee shop, or any of the other non "work" related areas of our headquarters and make these same statements. Somehow, there is a strange fear of "real time" awareness and communication that still hampers people's ability to know what they should or shouldn't share with their coworkers.

More importantly, one of the "channels" in yammer was about video gaming. On this channel, there was quite a bit of activity (especially by those in IT). I think this is GREAT! I think video gaming is the future of user centric software design. How much more output could we get from our people if we could get all our business applications as engaging and addictive as tetris or freecell. How much more of their free time would they be willing to donate to our cause if it they enjoyed their work as much as their video games? More importantly, how much more connected to each other are our employees and customers if we are personally engaged in conversations about things we care about?

What happened though, was that someone, somewhere was shocked! SHOCKED I say! that an employee who spends a mere 60 hours per week at the office has time to write a two line blog entry, three times per day about "video games". One of the participants in the "video game" crew yammered a chilling message to the group about how "we are being watched" and you should only talk about work related things.

The problem right now is where does work stop and fun begin? In the ideal world they completely overlap and you are not a split personality, but a whole person who get paid to do what they enjoy.

Friday, May 22, 2009

Some Folks are Unreachable

We have a perennial debate at work (going on 3 years now) about the "proper" and official Source Code Control tool in our company. Originally it was a crappy tool called CCC Harvest (yeah, sue me, it sucks I can't say it any other way), we then had a brief renaissance and used CVS (not great, but a big productivity boost!), then moved to Clearcase (shoot me now, it has a lot of potential and it theoretically GREAT, but in practice sucks almost as bad as Harvest), then began the migration to SVN (whopee, next step Hg!).

Unfortunately, every step of the way has been met with irreconcilable folks in the change management area who seem to think that developers are simply a whiny bunch of prima donnas who are trying to rob the company blind. I say this because I keep hearing about the "superman" exploit as a reason we should use Clearcase. In case you don't know, the superman exploit is where a clever developer sneaks code in to steal money by taking money that is lost due to rounding errors and deposits it in their personal account.

First off, unless someone was criminally negligent in setting the tool up, this would be nearly impossible in SVN. In fact, it probably MORE difficult in SVN than clearcase. I say this because it is possible (and easy) to administratively erase things in clearcase that are very difficult (by design) in SVN.

What really burns me though is that every time we attempt to have an objective discussion about the relative merits of either tool, the change management team spends their effort spinning things instead of really helping make them better. They are a perfect example of a team that has totally forgotten who their customers are and why they exist. Worse yet, they refuse to acknowledge that the complaints of the development teams are even worth addressing.

ALERT! EMERGENCY FIX REQUIRED!!!

I'm taking a vacation day today. I woke up at 5:00 to take my daughter to school (trip to six flags), then went downstairs to put some finishing touches on some html I was fiddling around with to help with some new screens.

While doing this a developer on my team started IM'ing me in a panic about a production problem that they where going to do an emergency deploy to fix. I attempted to calmly ask her to please explain the exact situation to me, but it came out more like "ARE YOU F'ING JOKING!? TELL ME YOU AREN'T SERIOUS!, WHAT THE HELL?".

Whoops, no more coffee for me, my bad.

So then I opened my email inbox and discovered no less than a dozen emails about about how the day before there was some sort of error that cropped up based on a specific data condition and stopped an extract process dead in it's tracks.

For some reason, this software has run without exception for 2 months and suddenly yesterday at 5:00pm this data condition started to happen. What puzzles me is that this tiny miniscule detail seems to have eluded everyone involved. Nobody seems to have stopped and asked "Hmmmm, WHY did this start happening? What changed? We all just started digging to get ourselves out of the little hole we found ourselves in.

Some fixes I heard where "patch the the code to convert nulls to 0 or -1" or "update the production database and change the nulls to 0's or -1". What I didn't hear (still waiting by the way) is "Hmmmm, I wonder what changed to cause this to happen?" and worse yet, what I didn't hear was "I wonder what we'll break if we suddenly change all this data to a different (invalid) value?"


Now that I'm a bit calmer I'm also asking "what is the business impact to this?" From what I can tell, we had a "red alert all hands on deck emergency" response to a relatively trivial problem. I'm glad everyone want to help and react quickly, but in my experience reacting quickly without thought (Oh, your arm hurts? cut it off!) more often than not has some serious side effects.

Now, someone might come back with an explanation about why this particular problem garnered the response it did, but I'm still waiting on that one. It's now 10:00am and I'm ready to take a break....I think I'll go outside and mow the lawn.

As a followup, we ended up on a conference call about this and it turns out the problem was caused by a bad data condition that really shouldn't have happened. In this case it is probably good that it blew up because we now can figure out how people are putting data into the system that is incorrect. While it's not ideal that we had such a bit TODO, it is good that we didn't put in one of our "fixes". They would have masked the real problem and caused even MORE bad and incorrect data to be put into the system.

My learning from this is that sometimes I need to sit everyone down and keep asking questions. There are a lot of people who are in hammy mode and sometimes someone needs to sit down with them and help get calm. It's funny because I have a reputation as a hot head, but it usually takes quite a bit to work me up if everybody's being reasonable. I took the ARSE test a while back and I'm not TOO bad (yet).

Saturday, May 16, 2009

Meetings gone bad

Below is a mostly fictional schedule that mimics a pattern I see in my office (names and places are fictional, just illustrating a pattern).

8:00am-9:15am Daily Status meeting (room B342) - In attendance: Susan, Bob, Bill, Steven, Mike. Review where we are in the project... Bill is going to fix bug 123 Steven is going to set up server, Mike needs to review the Spec doc and make recommendations, Susan needs to catch up on emails.

9:00am-10:00am Project Planning meeting (room B231) - In attendance: Susan, Bob, Bill, Steven, Mike (all late), and Jim. Plan the work for this week... Bill is going to fix bug 123 Steven is going to set up server, Mike needs to review the Spec doc and make recommendations, Susan needs to catch up on emails. Jim needs Mike to get with him and explain how to use the new Foobernator component. Mike agrees to do it in the afternoon.

10:00am-11:00am Requirements Review Meeting (room B243) - In attendance: Susan, Bob, Bill, Steven, Jim. Review the requirements documents for the upcoming project. Find a few spelling errors (huh?) Move some paragraphs around... uncover some issues that need to be figured out. Bob and Bill are going to get back with clients to figure out what we should do about the issues. Agree to meet tomorrow to re-review the doc.

11:00am-12:15 Issues Meeting (room C333) - In attendance: Susan, Bob, John and Jim. Review "Issues list" to determine what we should be working on for this release. Attendees realized they need Mike and Steven so they pull them into the meeting.


The anti pattern is what I'll call the Night of the living dead. You know this is happening if you see herds of people moving from one meeting room to the other. They meander from meeting to meeting eating at the brains of folks... slowing turning everyone into lethargic zombies incapable of anything useful.

If you have 4 meetings per day with the same 6 people, get a room! No, really, I mean it, book a conference room and move all six people to the same place. I realise we're all overweight and need the exercise, but walking around with the same horde of people all day is a huge time-waster.

In addition, if you have technical staff that you rely on to do things, you cannot invite them to 8 hours of meetings per day and expect them to get anything done. I have literally (not recently) attened continuous meetings where at 9:00am I was given a task, and asked ever hour or so if I had completed it. This was being asked by people who where continuously in meetings with me!

My personal solution to this problem I have implemented over the last year or so has been to start being VERY aggressive about meeting attendance and internally putting a value on my time. Too many people seem to use status meetings as a warm blanket to protect them from the harsh reality of "getting things done". They feel safe in spending 50 hours per week replying to email and attending meetings, and have wondrous excuses why things don't get done. Fact is, many folks could be a heck of a lot more productive if they would just do the work at hand instead of Talking about
doing the work at hand.

In addition, there are people who love to Hijacks meetings to further their own personal agenda. Typically this happens when the person organising the meeting hasn't clearly stated the agenda so everyone makes up their own. Very entertaining to watch, not too entertaining to participate in unless your goal in life is to be a member of the PFD.

Sunday, March 29, 2009

Commander's intent

This is an enabling concept and should be a key component of effective military (or business) doctrine. If every commander in the military had to ask the boss what they where supposed to do when they encountered an unspecified problem, how effective would they be?

Imagine the Tank Commander on halting his tank and radioing to the platoon leader "hey LT, there's a building in my way, what should I do?" and then the platoon leader radioing to the company commander... all the way up to the president. Wouldn't be a very effective fighting force would it?

The idea of "Commander's intent" is that it relies on subordinates having an understanding of what the overall mission is and how they fit into it. It specifically frees subordinates to act in accordance with their understanding of what they are being asked to accomplish. In essence, the commander formally recognizes that specific details are the responsibility of the lower echelon and it the the lower echelon's responsibility to make decisions about them. In our previous example, the company commander should have said "we need to get our tanks to the river. Responsibility for the specific maneuvers that are required to get there is in the hands of the individual tank commanders".

This is an effective tool, when used properly, that allows organizations to free themselves from the logjam of "lookup up" the hierarchy to discover what should be done. In essence, leaders at all levels have the responsibility for making their vision understood to those below them. They should resist the urge to constantly adjust the exact implementation details. In many ways, this is also way to reduce the amount of (micro)management that is necessary to get a job done and frees leaders to accomplish more strategic or visionary tasks.

I apologize to any tankers who might be offended by this silly example and I realize the terminology might be off, but hopefully the metaphor made sense.

Friday, March 27, 2009

Kmart.com has a stretchy layout now!

Update from previous blog post about retailers and their non-widescreen compatible layouts. I was commenting on how nobody seemed to get how to properly design a web page so it could be stretchy... kmart.com has done it!

Congratulations guys! I also like the new experience, it really looks nice. I will say the left nav is a little disconcerting. I was initially confused when I clicked on the department and the popup disappeared and showed the details down below. It took a few seconds to understand what the heck actually happened.

Do the right thing...

I recently sat down with a group of technical people and I made a statement that now haunts me. The statement was:

"If a developer doesn't know exactly what they are supposed to do, they should just GUESS what they think should be done and move on. "


Once the words left my lips, a technical team lead at the table paled and began to argue about how that statement was totally incorrect. I was puzzled for a moment, but then realised this person probably thought I was implying that they should not EVER go back and ask what should be done. In addition, I think they heard me imply that developers shouldn't ever ask questions in general.

This was not really what I intended to communicate and in retrospect, my choice of words was unfortunate. To clarify, I obviously think people should ask questions, they should openly challenge requirements that are vague, ambiguous, or contradictory. More importantly, they should be free to use their best judgement in the absence of specific instructions. But if it is going to take 24+ hours for a business client (or the boss) to get around to answering your question, make an educated guess as to what you think they MIGHT say, make a note for future generations, fire of the question asynchronously, and continue to work toward a solution.

I'm not suggesting that this is ideal, but nothing ever really is ideal. As an example, a better situation might be that the business client or representative user is sitting within arms reach of the developer and they can simply answer any question immediately. If this is the situation then of course: ask away, have a rousing discussion, argue about it. My point is more that software development is a partnership and having the development team simply delegating every decision to their business partner (or the boss) is a sure fire recipe for disaster. In addition, it will very likely lead to a buffalo problem

In hindsight and to clarify, I think my statement really should have been:
"In the absence of specific instructions, coworkers (peers, subordinates) are authorised to exercise professional judgement to select the best course of action without fear of reprisal"

Friday, March 20, 2009

Buying a new keyboard via Bricks and Clicks

OK, I've had it with my old clunkly Dynex keyboard. Normally my typing speed is somewhere up around 80+ wpm, but with this keyboard it sinks down to around 70wpm... that's over 10% and just not acceptable when I'm knee deep trying to get ideas out of my head and into a computer before they disappear.

So I thought what I'd do is go online and see which local retailers have my product, and, more importantly, which one's had my product in stock in my local store. The only reason I would go to a Brick and Mortar site for electronics is because either:
  1. I want it NOW!!

  2. it's so expensive/heavy that I want to make sure I have a real person to yell at if something goes wrong



Off the top of my head I know I picked these stores due to their proximity and my inclination that they may have what I'm looking for.
  • Best Buy (800lb gorilla).. I have a bias toward them because it is very close to my house and I know they have keyboards I can test drive.

  • Wal Mart (close by, cheap, I THINK they have keyboards I can test drive)

  • Kmart (close by, I work for their parent company, I'm pretty sure they don't have them out where I can test drive them in the brick and mortar)

  • Staples (far away, expensive)

  • Office Max (far away, expensive)

I put the broadline retailers in the middle because my impression of Office Max and Staples is that they typically charge a premium, but I don't know if that's really true or not.

First, the raw results:
  • Best Buy results (A) 510 results, many computer keyboards, the actual type of product I'm looking for is dead smack in the middle of the screen

  • Walmart results (D) 164 results, some computer keyboards, have to scroll down quite a bit to find a computer keyboard among the chaff

  • Kmart results (A) 238 results, many computer keyboards, almost everything is computer keyboard related. Actual product is slightly below middle of screen.
  • Staples results (B) 156 results, many computer keyboards, single list, only two results immediately visible

  • Office Max results (B) 102 results, many computer keyboards, single list, only two results immeidately visible


Now for the big test. I want to now filter by things I can go and pick up right NOW.

  • Best Buy (F) there might be a way to do this, but after a minute or two, I couldn't find it
  • Walmart (A) - OK, I hate to say it (for obvious reasons given my employer) but these guys nailed this cold. Not only does it give me the UPC (which Is interesting to me for other reasons) it also tells me which stores in my area have it. Nice job guys.
  • Kmart (F)
  • Staples (F)
  • Office Max (F)

Well, this was disappointing for me. I fully expected it to be a no brainer and that Best Buy would blow me away, and for some reason, I felt certain kmart.com had this functionality, but it was Wal Mart who actually came away the winner.

Frankly, this is a bit of an academic exercise because, after thinking about it, I'm still going to Best Buy because I want to be sure I can test drive the keyboard and I know they have a big display of them set up for me to play with.

Saturday, February 28, 2009

Motif ES 8 and linux

Last year (or maybe two years ago) I purchased a Yamaha ES8 workstation. It's been a great tool, although I just started using it last week after about 4-5 months of disuse. One thing I've been wanting to do was start editing/mixing and otherwise integrating the workstation with my PC.

The big problem is that I run Ubuntu linux at home. On the yamaha site, their software appears only for windows and Mac. Since I a big geek at heart I thought I'd see how far I can get anyway.

For my first step, I thought I'd plug the Motif into the USB port and see what happened... Well, nothing. lsusb didn't even return it as a device. However, dmesg reported a new device.

[67560.034283] usb 2-2: new full speed USB device using ohci_hcd and address 2
[67562.707240] usb 2-2: configuration #1 chosen from 1 choice
[67562.803313] usbcore: registered new interface driver snd-usb-audio

So, just for grins, I restarted the ES with the USB already connected. This seemed to allow the usb subsystem in linux to now see it. lsusb now returns:

Bus 003 Device 004: ID 045e:006a Microsoft Corp.
Bus 003 Device 003: ID 1267:0103 Logic3 / SpectraVideo plc G-720 Keyboard
Bus 003 Device 001: ID 0000:0000
Bus 006 Device 001: ID 0000:0000
Bus 005 Device 001: ID 0000:0000
Bus 004 Device 001: ID 0000:0000
Bus 002 Device 003: ID 0499:1023 Yamaha Corp.
Bus 002 Device 001: ID 0000:0000
Bus 001 Device 004: ID 0bda:8197 Realtek Semiconductor Corp.
Bus 001 Device 001: ID 0000:0000


So it seems we're now getting somewhere. In order to start easy, I thought I'd boot a windows virtualbox session and add this usb device to see if I could you the "out of the box" yamaha software.

So I downloaded the full fledged virtualbox deb from Sun. I already had the open source virtualbox that is included with the ubuntu 8.04 repos installed and I needed to uninstall this before installing the full version. After doing this, I went to the USB tab and added the detected yamaha device to the options.

When I booted into my XP virtualbox session, it detected the new hardware, so I tried to autoinstall. It didn't work so I clicked finish and went to plan B which is to install the software manually.

So I unzipped the usb midi driver to a temp directory, went through the add new hardware wizard and rebooted my machine. I then proceeded to start installing some of the other software I downloaded from the yamaha web site.

First off the voice editor (version 2.2): Being adventurous I tried to install from withing the zip file without first unzipping. First problem I noticed after installation is that it appears to be for the ES6... not sure if that's a problem, but the bigger issue is that is didn't appear to work. My start menu has a broken link to the voice editor user manual, but not software.

A quick search of the web indicates I should have followed the directions (or read them) as it appears the part editor requires the studio manager. Next, hoping I hadn't already screwed things up, I installed that.

On a side note, I discovered I needed to install acrobat reader as all the manuals are in PDF. It seems like a nice option might be to export manuals as html for folks who are viewing them and not printing them. PDF is just kinda sucky on a widescreen monitor and a 33mb download to read a text file seems a bit excessive to me. On another note, the default acrobat installer installs some sort of download manager, just what I needed, more stuff to go wrong on my pc. Well, at least it's just a virtual image and I won't lose stuff outside my VM.

Back on track though I soon discovered I had installed an ancient version of the USB MIDI driver. I had to uninstall the old one and reinstall the new one.

Well, this is as far as I could get so far. For some reason my virtualbox copy of XP cannot interact with the yamaha MIDI device. I think I may send a note to the yamaha folks to see if they can give me a hand and I'll also try to read the F'n manual to see if I missed a critical step.

I'll keep you posted as updates happen.

Sunday, January 18, 2009

Big three retail web stores

First off, let me confess, I am employed by Sears Holdings Company, so my evaluation might have some bias. I have tried very hard to be as unbiased (for OR against) as possible. That having been said, one thing that really bothers me is how many large retailers with an online presence continuously fail to understand basic things like web design.

As a quick and unscientific measure, I thought I'd review a big box retail sites and give a quick once over. The selections are based on my personal preferences and have no real scientific backing. Let's use the following sites:
Wal-Mart
Sears (obviously)
Target

To make it real, I'm actually going to try and buy some new socks. It's winter here in Chicago and my last pair of Smartwool socks now has a hole in it. Again, not scientific, pretty practical.

First off, the 800lb gorrilla of retail.



One thing that blows me away is the huge waste of real estate for anyone with a wide-screen monitor. If you look at this image, the yellow and black hatching shows how much wasted space there is. I've spoken with some folks working in online divisions of other retailers and all I ever hear is "but it's HARD to build a page that fills different monitor sizes". My Answer "Yep, that's why competent web designers and developers get paid lot's of money".

Beyond that, there are advertisements from third parties hawking car insurance (apparently from two different companies). It's pretty cluttered and seems to have no rhyme nor reason to how things are organized. Overall I'd say my impression is that it's design is on the borderline of professional and hobbyist.

Frankly, the site is cluttered, crowded, and otherwise a cacophony of sights and sounds reminiscent of the days before people started paying attention to basic design tenets.

In addition to the clunky design, the menus seem to pop underneath the advertisements when using Firefox. The Marketing guys at Wal-mart's online division would probably be a little upset if almost 1/2 of their potential customers will have a crappy experience. Why is this simple thing broken? I bet it 1/2 the people at a Wal-mart got hit in the head with a brick when the entered the store it would get fixed quickly.



Now I search for "wool socks" and get a good 20-30 pages of ever kind of sock EXCEPT wool. Cotton, nylon, aluminum (just kidding), but no wool. Only at the VERY bottom of the page do I get results for wool socks, but they're from a different retailer.



OK, Walmart, you get an 'A' for honesty, but an 'F' for usefulness. If I wanted general purpose search, I'd use google and never bother with your search tool.

Now for Sears.

Sears has the same wasted space problem as walmart. It's as if the online retailing giants all got together and decided to ignore wide screen monitors. Sears, on the other hand, looks nice and tidy and it seems they've spent a lot of time making thing work together. In addition, it doesn't have the apparently random advertisements that appear on Wal-mart's site. Sears works and looks nicely in Firefox.

Search blew me away, it displayed a sane number of results and they where ALL wool socks, exactly what I was looking for! Or so I thought. It turns out I assumed they where all wool, but nowhere on the page did it explicitly say that. If I had my way, I ask to have the search term displayed in context with the results.



Next, Target.

Huge waste of screen space again, all three so far all pretty have this same problem. Target also looks clean, uncluttered, and well organized. The bright red is a striking contrast to the cool blue for both Wal-mart and Sears. I suppose this could work for or against it. As far as Firefox, Target seems to work as well as Sears.

Search was wholly disappointing, Target returned only 1 result for my search which, instead of being socks, was a book!


For a company that uses a bullseye as their logo and a name like Target, they sure missed this one BIG.

Overall, I'd say the results are pretty surprising. For a company as large as Wal-mart with a veritable license the print money, it's web site is pretty clunky. It's search is antique and off-mark and the clutter of the extra unrelated advertisements just makes the site seem unfriendly and cluttered.

Target, on the other hand, looks very nice and clean and seems to function well. The inability to return ANY relevant results is a serious miss though. They probably need to go find a real search engine and properly index their site.

Sears, though, was a clear winner. The site was clean, functional, and it's search results where perfectly on the mark. My only improvement would be to make sure that the relevant search terms could be mixed into the list of results to make comparison shopping a little easier. Good job Sears.com!

Like I said before, I happen to work for Sears Holdings (though not in in the Sears.com division) but I've tried to be as unbiased as possible. I haven't been paid for this and frankly thought that Sears was not going to perform as well as it did.

If you think my findings are suspect, try for yourself and form your own opinion. I would love to hear your personal results!