Wednesday, April 16, 2014

Continuous integration versus delayed integration

A vigorous area of debate in the development and architecture community exists around the value of Continuous Integration. To give context, when a software development team gets to a certain amount of concurrent work that involves multiple teams making changes in the same codebase, there are two main ways to handle this:

  • Each individual stream of work take a copy of the code and start working on their own
  • Each team coordinate all work streams within the same codebase

In my experience, the former method seems like a great idea, especially when folks start worrying about "what happens when project A impacts project B's timeline?". Everybody just works on your own feature and at the end of each team's work stream, just merge all the individual team's efforts back together in a shared location. The great advantage here is that if Feature A is pulled from the delivery, Feature B can be deployed independently. What is frequently overlooked is the fact that eventually, Features A and B still need to be mixed together. The longer we wait to do this, the greater the risk of integration problems and the less time we have to resolve them.

On the other hand, having everybody pushing work into the same development stream immediately introduces some problems that need to be resolved at build time. It's immediately necessary to have a process to allow reconciling competing Features and disable features should it not be complete before delivery. The major downside continuous (or at least frequent) integration is that individual developers and Feature teams are a bit slower day to day as they uncover unanticipated problems due to competing requirements in the various work streams, but the upside is that these are discovered early in the process instead of at a very late stage.

In short, delayed integration doesn't actually save any time in the long run as the will eventually still manifest themselves, just later in the process when you have less time to fix them. To all folks doing multi-stream development, seriously investigate limiting your work to "one branch per environment", and work out how to independently configure the environments to know which features they should enable.

Friday, November 22, 2013

Web application framework popularity over time

I thought I'd do a quick comparison of web frameworks to see how popularity is trending nowadays per google trends. I realized this is has some serious drawbacks, but as a person who has been in the j2ee space for years, I find the trend very interesting.

The trend is clear, if you're an ace J2EE super guru, you are about in the same position as a yii developer. While you're experience might be transferrable, the days of java containers ruling the universe appear to be numbered. To do some other interesting comparisions, let's look at java specifically.

This roughly supports my observation that Struts is dying off, spring is holding stead, and gwt...while making a big splash a few years ago seems to be tapering off. These numbers are a little deceptive because back in 2005 a lot of the "up and coming" frameworks, languages, and tools didn't even exist. So let's take a look at just the upstarts.

I've included rails because it's relatively new, and didn't such a large downturn and slow slide into oblivion as struts even though they started in a relatively similar timeframe.

For an up to date view of current programming language trends (not just frameworks), take a look at the Tiobe Index. It's interesting that C has been the top language for 40 years and shows now sign of faltering, whereas something like COBOL peaked 20 years ago and has long been in decline.

Use this information as you will, but some comments I will make:

  • Popularity is important, frameworks and languages need a vibrant community
  • Search trends can lie... something with high search requests could just suck and therefore more people need to look things up
  • The days of the software monoglot being in high demand (think J2EE developers in 2004) are rapidly receding. On one hand, the market is growing so the aggregate job demand might be the same. But from a solution perspective, there are many many more options now than there were even 10 years ago.

Running off the jetway or how to make decisions under pressure

The hazards of the unknown unknowns or how to avoid Running off the jetway

This scene illustrates an all too common problem in any field, and is one that I've encountered over and over again. We often entrust major decisions to folks who don't have enough information or are working with major assumptions about the situation that are incorrect. In this example, Jim Carrey's character is running furiously to get a briefcase back to it's owner, and when stopped (because the airplane has left), he assumes that his status as a Limo driver entitles him to board the plane even when no one else is allowed. The critical piece of information he was missing was that it wasn't simply his lack of status that was the reason he couldn't board the plane, but the fact that the plane had already left.

This is a lot like many computer 'experts' who claim that because of their status, they can ignore warnings or requests from others because 'They know better' about this sort of thing. They have the opinion that they are knowledgable enough to make broad assumptions about 'how things are supposed to be', but often are missing important and dangerous details. These blindspots are made famous by Donald Rumsfeld as unknown unkowns and cause many problems that we end up spending a lot of time extracting ourselves from.

To avoid the pain of running off the jetway, there are two important important things to remember:

  1. Don't Panic
  2. Be a good listener

Dont Panic: because once your brain goes into fight or flight mode, you can make bad decisions. This means you will revert to "Type 1" or automatic thinking. Your decisions will be short sighted and even irrational because your brain is busy trying to save you from whatever the emergency situation happens to be.

Be a good listener: since often the details of the situation can illuminate important details that make things much less urgent than they might seem on the surface. It takes careful and active listening skill to tease these details out and they will be buried if everyone is busy broadcasting their opinion. Calm ourself, listen to what is going on, and most importantly, start to detect what you aren't actively looking for.