Cavemen, Exploding Nails, and Software Development
I've work now for a number of years in organizations of various sizes producing software. Having worked to produce wildly successful and effective solutions as well as hugely expensive boondoggles that are useless I ponder why sometimes things work out and sometimes they don't.
Many folks claim that Software development is in it's infancy and this is the reason for the erratic results. If we're using human development as the metaphor for software development, then I think the current state is not even quite human yet. I think our state is somewhere more along the lines of a caveman or some other ancient common ancestor.
Where we think we are
Where we really are
Or maybe here
What makes things difficult is that people try to map processes and activities into a physical space. We talk about software as if we are trying to build a bridge or a house. Then we take processes used to perform these activities and try to shoehorn development processes into them. Sometimes this actually works and sometimes it doesn't. When works, it then gives people the impression that they have found the solution for performing this activity.
For some reason though, everyone in the has been ignoring the fundamental problem with using these metaphors. In most physical world systems, components seem to behave in a linear fashion. What I mean is that when building a house, it is highly unlikely that ONE nail in the wrong place will cause the house to explode. Nor is it possible for one person to drive all the nails for the house in 1 minute. In software however, this is entirely possible and it happens all the time.
It is interesting because this is actually a fundamental reason why software is so useful. A small stimulus can yield a largely amplified effect. To use the house building metaphor, imagine we could build a magic hammer that could drive every nail into place simultaneously. Most folks would say that would be impossible, but in software development, no problem.
Let's give an example of software that does this. Right now, go find a web browser, navigate to www.google.com, and search for "new york times". In addition to being able to jump into the actually newspaper content, you will also find thousands (or maybe millions) of books, articles, blog entries and other content that is related. If we where to take all this and map it into the physical world, you would have a book that weighed about 1.2 billion pounds and was 10,000 feet tall. (creative.cloud)
Imagine building a physical system using anything other than computer software to be able to search this content and yield results. Now image making that system work (remember, no computers) in less than 1 second. Now further imagine making that system work so that you didn't have to move from your current position nor do anything except type three words (onto a typewriter I'm assuming).
With this background, I'll now get to my point: Because software development is such a large amplifier, it is really easy to produce extraordinary results. Sometimes the results are extraordinarily good, sometimes they're extraordinarily bad. The software used by google is probably pretty complex and the hardware is also likely complicated, but it is nothing compared to the RESULTS.
It isn't that software development is immature, it's that it's not yet the right species for some of the the types of activities we're trying to use to control the process. It would probably be difficult to teach a precursor species to play baseball if they had no opposable thumbs (even if they possessed the intellect).
Many software processes and organizations at this point are designed to limit the variation of the input in order to achieve predictable outputs. This is a dicey proposition at best as software because of the nonlinear amplification effect that software provides. What we need to develop are processes and strategies that use the strengths of software to amplify the probability of producing positive results.
To this end, I think our answers lie well outside traditional engineering disciplines. The fact that engineering of physical systems is inherently limited by the reality of the "real world", we need to start looking elsewhere for our solutions. I think fruitful areas to look for solutions are going to be in areas like psychology and sociology. While writing software, the most important factors are not the software languages, tools, or other documentation we constantly worry about. The more important factors are evolution, communication, motivation, and social dynamics.
While it's true that the latter factors impact any endeavor, most traditional systems have a natural a damping effect on their impact. Software development, being "all in our heads" doesn't have this limitation and is therefore more sensitive to their effects.
Many folks claim that Software development is in it's infancy and this is the reason for the erratic results. If we're using human development as the metaphor for software development, then I think the current state is not even quite human yet. I think our state is somewhere more along the lines of a caveman or some other ancient common ancestor.
Where we think we are
Where we really are
Or maybe here
What makes things difficult is that people try to map processes and activities into a physical space. We talk about software as if we are trying to build a bridge or a house. Then we take processes used to perform these activities and try to shoehorn development processes into them. Sometimes this actually works and sometimes it doesn't. When works, it then gives people the impression that they have found the solution for performing this activity.
For some reason though, everyone in the has been ignoring the fundamental problem with using these metaphors. In most physical world systems, components seem to behave in a linear fashion. What I mean is that when building a house, it is highly unlikely that ONE nail in the wrong place will cause the house to explode. Nor is it possible for one person to drive all the nails for the house in 1 minute. In software however, this is entirely possible and it happens all the time.
It is interesting because this is actually a fundamental reason why software is so useful. A small stimulus can yield a largely amplified effect. To use the house building metaphor, imagine we could build a magic hammer that could drive every nail into place simultaneously. Most folks would say that would be impossible, but in software development, no problem.
Let's give an example of software that does this. Right now, go find a web browser, navigate to www.google.com, and search for "new york times". In addition to being able to jump into the actually newspaper content, you will also find thousands (or maybe millions) of books, articles, blog entries and other content that is related. If we where to take all this and map it into the physical world, you would have a book that weighed about 1.2 billion pounds and was 10,000 feet tall. (creative.cloud)
Imagine building a physical system using anything other than computer software to be able to search this content and yield results. Now image making that system work (remember, no computers) in less than 1 second. Now further imagine making that system work so that you didn't have to move from your current position nor do anything except type three words (onto a typewriter I'm assuming).
With this background, I'll now get to my point: Because software development is such a large amplifier, it is really easy to produce extraordinary results. Sometimes the results are extraordinarily good, sometimes they're extraordinarily bad. The software used by google is probably pretty complex and the hardware is also likely complicated, but it is nothing compared to the RESULTS.
It isn't that software development is immature, it's that it's not yet the right species for some of the the types of activities we're trying to use to control the process. It would probably be difficult to teach a precursor species to play baseball if they had no opposable thumbs (even if they possessed the intellect).
Many software processes and organizations at this point are designed to limit the variation of the input in order to achieve predictable outputs. This is a dicey proposition at best as software because of the nonlinear amplification effect that software provides. What we need to develop are processes and strategies that use the strengths of software to amplify the probability of producing positive results.
To this end, I think our answers lie well outside traditional engineering disciplines. The fact that engineering of physical systems is inherently limited by the reality of the "real world", we need to start looking elsewhere for our solutions. I think fruitful areas to look for solutions are going to be in areas like psychology and sociology. While writing software, the most important factors are not the software languages, tools, or other documentation we constantly worry about. The more important factors are evolution, communication, motivation, and social dynamics.
While it's true that the latter factors impact any endeavor, most traditional systems have a natural a damping effect on their impact. Software development, being "all in our heads" doesn't have this limitation and is therefore more sensitive to their effects.
Comments