Sunday, December 9, 2007

Why experts suck

I'm reading "supercrunchers" and it has reconfirmed something I've thought for a long time. Experts suck.

Time and time again I find myself at odds with authorities in varying disciplines. Usually the situation develops when we have a problem, consult the experts, and they postulate that the problem is something that can only be solved with a very big and very expensive project.

I will spend some time, form some models, run some analysis and reach some alternate conclusions. I am met by a stubborn and braying pack of obstinate mules who have no argument against my data except that they are "the experts" and I should stop questioning their authority on this subject.

I have, to date, simply thought these people where just jerks and not worth my time. It turns out, however, that the problem is very widespread and many people seem afraid that if we can build a statistical model that can effectively do what their "expertise" provides, then they'll be out of a job. I need to start showing that if we free these folks from this menial task, they can then start doing the creative part of their job and let the computer do the remembering for them.

No comments: