Why software estimates change and inflate

-

As a software developer (Architect?) I find myself in a constant battle with clients and project managers about changing estimates and their inaccuracy. I've lost count the number number of times I've given an estimate and then had to revise it to the dismay of folks who assumed seemingly subtle changes would move the estimate down or allow it to remain the same, only to see it creep up. There are a number of variables that increase the estimation risk and I'll briefly touch on the major factors.

Major factors are:

  • Changing requirements/assumptions - a fundamental truth is that any change incurs overhead. The act of changing the design to remove a requirement is is work too...remember, even simplifying the design is work. Removing a requirement mandates revalidating a design against (even if they're simplified) the new requirements.
  • Changing the team structure - an experienced dev is much more effective than a newbie. Moreover a person well versed in a particular solution is often more effective than a more experienced resource who is unfamiliar with the existing code. Creating estimates for an unknown team is tremendously difficult and often leads to large inflations to account for the risk of getting an unvetted resource.
  • Work fills to accomodate available time - if you give a developer 40 hours to complete a task, they will almost always take AT LEAST that amount of time. Even if it seems to be a simple task, they will spend extra time to analyze options, test, and otherwise use available time even if they COULD have potentially "just done the work" in four hours.
  • Estimates are just starting points - the harsh reality is that estimates for non-trivial software are starting points and evolve as more information becomes available. The more analysis you do without obtaining more information, the higher the multiplier is that the estimate is based on faulty information (especially when it involves the factors mentioned above

The short version is that "all software development is design". Any change anywhere changes that design and thus creates more work. Agile proponents realize this (maybe implicitly) and combat this problem by locking down design for periods of time to help move things forward (with real deliverables). Long drawn out design cycles cause extra work that too often is underwater.

Comments

Post a Comment

Popular posts from this blog

Please use ANSI-92 SQL Join Syntax

-
Image
Way back in 1992, a standard was published for SQL. In it, the syntax for specifying joins in SQL Database was finally standardized. This was a good thing because (for folks who've been doing this a while), trying to decode the vagaries of how various Databases handled outer joins could drive you bonkers yielding different results depending on the parse order of things in the from and where clauses. Unfortunately, some DBMS vendors took a while to adopt this standard (Oracle didn't support it until 2001), but at this point if you're using a major RDBMS and NOT using it, you're probably mired in a tradition or habit that bears changing. As an example of what I'm describing, here are some examples of what I mean. Old School (no standard syntax + implementation and evaluation of predicates varied by vendor [and query]): select * from customer, order where customer.customer = order.customerid ANSI way (agreed to standard, predicate evaluation has some rules ...
Read more

The difference between Scalability, Performance, Efficiency, and Concurrency explained

-
I thought I'd give some explanation of the difference between scalability, performance, Efficiency, and concurrency. To make things more accessible (and because I'm watching a cooking show on TV) I'm going to use a baking metaphor to help explain the differences. In our case, the system in question is baking cookies for consumers. Scalability, is how well something responds to adding more resources. If we can double our capacity by doubling the number of resources, we've got linear scalability. In our baking example, if we add more ingredients, we can scale to a larger number of cookies. To contrast with our other metrics, this doesn't actually increase the performance of creating a single cookie (they still take 10 minutes to bake), and the efficiency is the same (we just have more capacity). We also cannot bake more cookies in a fixed time period, but over a larger time period we can produce more cookies. Performance is how fast something can get done, u...
Read more

the myth of asynchronous JDBC

-
Image
I keep seeing people (especially in the scala/typesafe world) posting about async jdbc libraries. STOP IT! Under the current APIs, async JDBC belongs in a realm with Unicorns, Tiger Squirrels, and 8' spiders. While you might be able to move the blocking operations and queue requests and keep your "main" worker threads from blocking, jdbc is synchronous. At some point, somewhere, there's going to be a thread blocking waiting for a response. It's frustrating to see so many folks hyping this and muddying the waters. Unless you write your own client for a dbms and have a dbms that can multiplex calls over a single connection (or using some other strategy to enable this capability) db access is going to block. It's not impossible to make the calls completely async, but nobody's built it yet. Yes, I know ajdbc is taking a stab at this capability, but even IT uses a thread pool for the blocking calls (be default). Someday we'll have async databa...
Read more