Friday, November 22, 2013

Web application framework popularity over time

I thought I'd do a quick comparison of web frameworks to see how popularity is trending nowadays per google trends. I realized this is has some serious drawbacks, but as a person who has been in the j2ee space for years, I find the trend very interesting.

The trend is clear, if you're an ace J2EE super guru, you are about in the same position as a yii developer. While you're experience might be transferrable, the days of java containers ruling the universe appear to be numbered. To do some other interesting comparisions, let's look at java specifically.

This roughly supports my observation that Struts is dying off, spring is holding stead, and gwt...while making a big splash a few years ago seems to be tapering off. These numbers are a little deceptive because back in 2005 a lot of the "up and coming" frameworks, languages, and tools didn't even exist. So let's take a look at just the upstarts.

I've included rails because it's relatively new, and didn't such a large downturn and slow slide into oblivion as struts even though they started in a relatively similar timeframe.

For an up to date view of current programming language trends (not just frameworks), take a look at the Tiobe Index. It's interesting that C has been the top language for 40 years and shows now sign of faltering, whereas something like COBOL peaked 20 years ago and has long been in decline.

Use this information as you will, but some comments I will make:

  • Popularity is important, frameworks and languages need a vibrant community
  • Search trends can lie... something with high search requests could just suck and therefore more people need to look things up
  • The days of the software monoglot being in high demand (think J2EE developers in 2004) are rapidly receding. On one hand, the market is growing so the aggregate job demand might be the same. But from a solution perspective, there are many many more options now than there were even 10 years ago.

Running off the jetway or how to make decisions under pressure

The hazards of the unknown unknowns or how to avoid Running off the jetway

This scene illustrates an all too common problem in any field, and is one that I've encountered over and over again. We often entrust major decisions to folks who don't have enough information or are working with major assumptions about the situation that are incorrect. In this example, Jim Carrey's character is running furiously to get a briefcase back to it's owner, and when stopped (because the airplane has left), he assumes that his status as a Limo driver entitles him to board the plane even when no one else is allowed. The critical piece of information he was missing was that it wasn't simply his lack of status that was the reason he couldn't board the plane, but the fact that the plane had already left.

This is a lot like many computer 'experts' who claim that because of their status, they can ignore warnings or requests from others because 'They know better' about this sort of thing. They have the opinion that they are knowledgable enough to make broad assumptions about 'how things are supposed to be', but often are missing important and dangerous details. These blindspots are made famous by Donald Rumsfeld as unknown unkowns and cause many problems that we end up spending a lot of time extracting ourselves from.

To avoid the pain of running off the jetway, there are two important important things to remember:

  1. Don't Panic
  2. Be a good listener

Dont Panic: because once your brain goes into fight or flight mode, you can make bad decisions. This means you will revert to "Type 1" or automatic thinking. Your decisions will be short sighted and even irrational because your brain is busy trying to save you from whatever the emergency situation happens to be.

Be a good listener: since often the details of the situation can illuminate important details that make things much less urgent than they might seem on the surface. It takes careful and active listening skill to tease these details out and they will be buried if everyone is busy broadcasting their opinion. Calm ourself, listen to what is going on, and most importantly, start to detect what you aren't actively looking for.

Wednesday, November 20, 2013

Running and software development

I have a long love/hate relationship with running and I think that it's a great metaphor to help explain the subtle differences between agile practices and traditional development. My kids have been bitten by the running bug and they devote a lot of time to cross country and track. More importantly, they have trained for running longer distances and therefor better understand the importance of preparation, pace, and form.

As a person who spends a lot of time playing soccer, the idea of NOT running as fast as possible still requires a LOT of mental energy. On the rare occasion that I still get out running longer distances with my kids, they routinely tell me to slow down for the first mile... then they scratch their heads because I burned myself out blazing through the first mile in 7-8 minutes.

In software development, agile practices are the equivalent of the type of running done in soccer... you are actively changing direction, reacting to things on the field and using strong, explosive, but short bursts of energy. Scrum actually borrows metaphors directly from rugby to help explain the activities and practices (Rugby being a form of European football closely related to Association football from which Soccer derives its name).

More traditional linear development process and practices are more like running distances, where pacing yourself, conserving enough reserve energy to make the entire distance is more important than getting 40 yards ahead of the competition as fast as possible. Iterative or Agile processes are more like Football (soccer, rugby, american, austraian rules) where explosive movement and short timings are more important than endurance and predictable splits.

There is a place for both approaches, but too often folks want to try and do BOTH at the same time and it just doesn't work. It's a bit like trying to sprint a marathon while chasing a ball... it's just not going to work.

Monday, November 18, 2013

Using Olark online chat component

So I thought I'd investigate SAAS online chat components and I consulted the almighty google and selected my first contender. boldchat seemed like a good option until I discovered it seemed to need an exe to function. As I'm a max/linux snob, this immediately removed it from the running. My next contender was selected from a post on Stack Overflow was olark. I'll have to say that from the get-go, this seems like a very fully featured and easy to use tool. I was up and running within 5 minutes. After toying with this for a bit, I'm really quite impressed. If you need a quick and easy online chat on your web property, olark is a great way to get moving quickly.

Sunday, November 17, 2013

Fun with character encoding and when to use ISO8859 instead of UTF-8

What character encoding are you using? Most folks nowadays settle on UTF-8 for web centric type applications, but things can get squirrelly if you use this encoding and start working with non-unicode systems. Recently, we had a situation where we took the string representation that started out with an array of 17 0xff values. In a unicode aware system, using UTF-8, this will translate into a character sequence of 17 0xfffd values.

What happened? How did an array of 8 bit values get magically translated into an array of 16 bit values?

It requires a bit of digging, but the short version is that if your source system is using 8 bit characters (something like iso 8859-n) and you translate to unicode, you will fail on certain byte values because they are invalid in UTF-8 and other character encodings. The only thing that can then happen is to change the character to the "invalid character" which is 0xfffd. For reference, in UTF-8 the values 192, 193, and everything over 253 are invalid and will be translated into 0xfffd.

So what's the solution? If you MUST do this because you are interacting with an embedded device or something else that is not unicode aware, the simplest would be to use an ISO 8859 charset to support characters larger than 0x80 (128). This can be quite a challenge if you actually need to get the correct glyph because there are a number of charsets in this space. Note, in Java at least, all characters are 16bit values, so there is often some magical transformation necessary to switch between bytes and chars.

Examples for folks who'd like to see the problem in code:

char[] ba = {0xff,

byte[] ba2 = new byte[ba.length];
        for(int i = 0; i < ba.length;i++) {
            ba2[i] = (byte)ba[i];

        try {
            String ascii = new String(ba2, "US-ASCII");
            String iso8859 = new String(ba2, "ISO-8859-1");
            String utf8 = new String(ba2, "UTF-8");

            char asciichar = ascii.charAt(0);
//this will be 65533 (0xfffd)
            char isochar = iso8859.charAt(0);
//this will be 255 (0xff)
            char utfchar = utf8.charAt(0);
//this will also be (0xfffd)

Note, there are still challenges as a char is 16 bits and you'll need to be careful when traversing systems to let everyone know your output is in an ISO-8859 character set. For example, if you generate an xml file and set the encoding to UTF-8, the file will contain �����������������, but if you use ISO-8859 it will contain ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ.

A more entertaining aspect of this is if you subsequently try to insert this string value into a database that is unicode aware and assume your characters will fit into 17 bytes it will overflow the width because the ����������������� string of 17 chars is actually 34 bytes, but the ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ string can be represented as only 17 bytes.

In short, the space of ascii between 128 and 255 is treacherous territory and using UTF-8 only helps if everybody uses UTF-8, otherwise transliterating between the code pages can be quite an adventure.