Monday, December 17, 2007

Entropy or Energy?

We try so hard to get our IT infrastructure organized, but I wonder how useful that is. In a typical business environment, we see these high costs of software "overhead" and try as hard as we can to reduce the cost to zero.
The problem with this is that we got our original gain from technology, not necessarily from the advantage of the technology itself, but also from the chaos created by changing from one paradigm to another. It is almost like a nuclear reaction and is just about a difficult to control. Too many restrictions and the energy disappears... Not enough restrictions and ... well BOOM!!!
Finding the balance between the highest energy and the highest entropy is a difficult task in IT indeed.

Sunday, December 9, 2007

Why experts suck

I'm reading "supercrunchers" and it has reconfirmed something I've thought for a long time. Experts suck.

Time and time again I find myself at odds with authorities in varying disciplines. Usually the situation develops when we have a problem, consult the experts, and they postulate that the problem is something that can only be solved with a very big and very expensive project.

I will spend some time, form some models, run some analysis and reach some alternate conclusions. I am met by a stubborn and braying pack of obstinate mules who have no argument against my data except that they are "the experts" and I should stop questioning their authority on this subject.

I have, to date, simply thought these people where just jerks and not worth my time. It turns out, however, that the problem is very widespread and many people seem afraid that if we can build a statistical model that can effectively do what their "expertise" provides, then they'll be out of a job. I need to start showing that if we free these folks from this menial task, they can then start doing the creative part of their job and let the computer do the remembering for them.

Sunday, December 2, 2007

New Songs

I wrote some new music and realized and audio engineer might be helpful. I terribly confused at how to compress the dynamic range into something that sounds OK on a radio.

Saturday, December 1, 2007

Enterprise Scalability and Architecture

I've recently spent a lot of time explaining scalability and architecture to people in our organization and am being largely unsuccessful. I blame hardware vendors for this, they shove new technologies at unsuspecting infrastructure professionals without any high level explanation about the real advantages.
As my first example, I will talk about logical partitioning. Listen folks, this is not a scalability feature! Don't listen to the salesman, he doesn't know what the hell he is talking about. I will admit I can imagine some scenarios where this might be able to help you scale a collection of applications, but do NOT try and tell me that adding a new LPAR is somehow scaling my application. Also do not tell me I should buy a machine that is 20x more expensive because it's more "scalable".
The problem with machines that share disk, buses, cache, or other resources is that you need to manage who is allowed to use the resource at any given time. It is very dangerous to allow two different processes to modify a memory location at the same time so you need to manage this. This management is not free and as you add more things (processors) sharing resources the overhead typically increases at an order of magnitude that is greater than linear. Put another way, this means the system will scale at an order of magnitude that is less than linear.
No matter what manner of wizardry your hardware vendor has put into his machines, if you are sharing anything, you are going to incur overhead. This overhead will also increase as you add nodes that need to contend with each other. So any salesman who tells you otherwise is simply wrong.
The number one most sure fire super hot way to scale an application is to partition the data and/or work and do the processing on independent machines. This is the holy grail of scalability, it is a sure fire, can't lose proposition. It is hard to do and does require thought, creativity, and a good deal of elbow grease. It is, however, also a huge reward for the effort and is the ONLY sure fire path to scalability.

Thursday, November 29, 2007

You can't be a manager, you've never done project management

I recently noticed a position within my organization that looked like a dream job to me.

"The primary role of this individual will be transformation. They will provide a fresh view into IT technology strategy and will transform the role of the team and the capabilities of the individuals. They will establish, implement, and maintain a unified technology strategy for the organization. They will lead a team of approximately 10 senior IT specialists and will actively participate in the highest profile, most strategic IT initiatives in the enterprise. The Enterprise Architecture team is not only accountable for setting our technology strategy, but is also required to assist project teams in implementing the strategy."

As a person who has spent 5+ years trying to transform our organization from the bottom up, I thought I would be a perfect fit for this position. Who better to help make this transformation than someone who has "been there, done that" and already made huge strides toward getting the organization working together and creating innovative solutions. In addition, I had a proven track record of real solutions to problems using leading edge technology, while still accounting for the risks this strategy might incur. In addition, I coordinate 6+ teams and ensure they understand what we are trying to do. I also introduce innovative and disruptive technologies to break us out of our rut, as well as build consensus among our various teams. I figured I was a shoe in (if the position wasn't actually written for me in the first place).

I quickly sent a note to my management team explaining my desire for this position. Of course, I was quickly reassured that I was probably not a very good fit for this position. After all, it would involve project management, budgets, reviews, and all manner of other non-technical stuff that I had so often said I had no desire to do. Stunned, I had to stop for a minute and think about what I just heard. These points where listed far down the posting as ancillary duties that where required, but in no way did anything in the posting infer that this was the primary goal of the position (I believe it actually said these things would be about 10% of the required duties)

Somehow, though, these ancillary activities where the absolutely critical skills necessary to the position. In addition, because I didn't need to use this skills in my current position, I obviously must not possess them and why in god's name would anyone think I did?

It's an interesting situation, does this mean it's obvious to everyone except me I must not possess these skills? Hmm, maybe I've reached my professional max...

Thursday, November 22, 2007

On Infrastructure and agility

Running myself through a our software development and delivery process, I was struck with an interesting and always present problem. It's all fine and dandy to be able to deliver working, high quality code at a rapid tempo, but somebody's got the RUN it!
This problem of the dichotomy between operational stability and innovation sits at the core of every decision I make lately. On one hand, the servers must be able to run in a cost effective manner. We can't just gut the data center every year because some new cool thing has been introduced just because it's cool. On the other hand, we can't just keep using the same old technology we've been using for 20 years because we can torture it into solving any problem.
For folks who have been programmed to "keep the data center running", any new or disruptive technology is bad. End of story, no question, no doubt. This is further strengthened by the "fact" that almost any problem that can be solved with a new technology, can also be solved with an existing technology (given enough effort).
This problem is much like exposing someone who has grown up using a hammer and nails to a new problem (screws). Upon hearing about a requirement to start fastening things with screws, they expend all their energy pounding the screws with their hammer. When the existing hammer proves too difficult (i.e. after wasting energy on it), instead of buying a screwdriver (the new technology), they buy a bigger hammer. While a bigger hammer certainly solves the problem, it is only the best solution if you are unable to comprehend the advantages of using a screwdriver.
Continuing the metaphor, when you analyze the reasons for continuing to use the hammer, it actually makes sense. Some logical reasons for this behavior are: I already know how to use a hammer, I don't want to retrain on how to use a screwdriver. I don't have any place set aside to store the screwdriver. Now I only need to support one tool, addition a screwdriver will mean I may need to hire a screwdriver expert.
On the other hand, I can certainly understand why we don't want to run off and do a "Noah's Ark" situation in the data center. I don't know how many times I've had to reign myself in to avoid pushing a particular new technology just because it's cool and makes my job a lot easier. Obviously saving me 15 minutes per week has a definite cost savings, but my corporation may not thing my 15 minutes is worth $15 million plus a staff of six to support it (although I'd disagree).
Back to the screwdriver metaphor, the questions that need to be asked would be: Why do you think you need to use screws? What are the advantages? Do these advantages outweigh the disadvantages of continuing to use nails?
While the hammer and screw metaphor is a little tired, it really "hit's home" with the seeming illogic and stubbornness of some folks that seem thick headed when it comes to implementing new technology.

Monday, November 19, 2007

Selling good practices

After an eye opening weekend, my coworkers and I went about trying to figure out which things we could easily do to make our software better. In addition, we took on the responsibility to "sell" it to our management team. I was completely floored when we started talking to the managers and they started getting really excited about this stuff.

I totally expected people to start saying things like "yeah but, we can't do that here" and instead we where met with "wow, that's great, let's do it!". Shows how well I know people, so, now we're getting somewhere....

Sunday, November 18, 2007

No Fluff Just Stuff 2007

I attended No Fluff Just Stuff in Chicago this weekend. There is only really one word to describe it, "wow". The speakers where stellar, the format was very personal, and the camaraderie was extraordinary.
Coming from an organization that is still very waterfall where testing is a manual process that people slog through, it was eye opening that some very BIG companies where doing this agile sort of stuff on very critical products in super mission critical situations. I spoke with people who wrote embedded devices (pacemakers), avionics software, pharmaceutical software, and many other products who's criticality make my current system (a retail item management system) look like a rather big "hello world" project.
The biggest gain I've taken away from this was that getting the management team's buy in isn't enough. I need to go gather metrics about the risks and waste in our current process and simply implement fixes to eliminate them. The agile community happens to have a large bag of tools to help do this.