Tuesday, December 28, 2010

Making the MQ versus RPC decision

Among many software architects and pundits, Message Queue solutions have a lot of press about being a highly scaleable solution in comparison with RPC based solutions. From what I can see, the biggest problem with most comparisons is that they start with the premise that one or the other of these two approaches is superior and then spend time trying to make a compelling argument why they are correct.

I'm going to throw my hat in the ring on this issue and offer a high level guide for folks who don't have the time or energy to dig into queuing theory or debate with ivory tower architects about the issue. You'll note that scaleability is not even a factor. This is deliberate as scaleable and performant solutions can be built using either pattern. There is an interesting performance comparison that seems to indicate that the performance characteristics are very similar for both approaches. I WILL point out that simple http-based RPC solutions DO have fewer middleware requirements and are almost universally accessible. Furthermore, I'll point out that it is entirely possible that a real world solution may need both.

In any event, some strong indicators you'll likely need an MQ based solution include:
  • I need multiple endpoints to subscribe to my message
  • I need durable messages that can be delivered even when the message destination is not running

From my perspective, if any of these are necessary, then some sort of MQ technology is probably appropriate. Certainly, if you try to build a custom solution to either of those problems, you're highly likely to be wasting your time.

On the other hand, some equally strong indicators that an RPC solution is probably your best choice:
  • I need clients to get an answer immediately
  • I need a very small footprint and performance and simplicity are more important than reliability

The first bullet is most important above, synchronous message queuing patterns (e.g. request/response) typically come with a lot of baggage that may be hurting both your performance AND your scaleability.

I've worked with a variety of solutions using MQ series, JMS providers, and Biztalk, used both appropriately and otherwise. In every case I can remember, if we had used the above guide, we would have had a better chance of picking a the most appropriate solution and also arrived at the conclusion more quickly.

Sunday, December 26, 2010

My experience at Denny's and three rules for success

I recently went out to breakfast with my family at our local Denny's restaurant. We arrived around 9:15am and there was a pretty "interesting" line of folks waiting to be seated. In retrospect this should have been an indicator that something was wrong and things were going to be slow.

We finally got seated and our server promptly brought menus and took our order, then we settled down and began waiting for our food. While waiting at out table, at least 6 other groups came in after us, were seated, ordered, got their food, and left.

After about 30 minutes, the greeter actively started telling people that the food was going to take from 30-90 minutes to prepare and people stopped being seated. In addition, while the greeter was saying this to new customers, our server kept telling us the food would be out "in just a little bit". By 11:00am, I was pretty irritated because our breakfast had turned into lunch and all the other places to eat in the area were now open (note, Denny's was NOT our first choice).

Finally our food arrived, we ate, and I had the dubious honor of talking to the manager about what went wrong. It turns out they only had one cook... normally this is a problem, but then they had a bunch of people show up, everyone panicked, and things went downhill fast. In effect, the entire attitude was that "it's not our fault"... unfortunately, I didn't really care who's fault it was or wasn't, I cared that I sat around for 1.5 hours waiting for some food I could have prepared myself in about 30 minutes.

This brings me to my three rules:

#1 When things get out of hand, tell your customers immediately. They are NOT going to be happier if you make them wait longer wondering what the heck is going on. Notice I didn't say "if". That's deliberate, I know "stuff" happens, reasonable people will get it.
#2 When things seem to be getting out of hand, adding another cook might not help. What seems to have happened in our case is that a few other employees tried to help the cook and screwed things up. Everyone started getting irritated, arguments started, and things started to spiral out of control.
#3 Treat your EXISTING customers as good or better than new ones. If they had approached me 30 minutes into the debacle and said "we're running a little slow, are you sure you want to stay, it might be another hour" I probably would have left. Yes, I would have been upset, but not nearly as upset as after waiting for 1.5 hours (with two kids under 5 by the way). Note, wireless companies, satellite, cable, and other companies could learn something about this too. I'm tired of wonderful deals for "new kids" and me getting substandard service because I'm already "hooked".

Wednesday, December 22, 2010

Rails, Grails, and convention over configuration

Back in 2003 I wrote a quick application generator using turbine, java, and xml and published to sourceforge called thrust. It is extremely primitive by today's standards, but the important point is that is embraced the "convention over configuration" concept.

I now see a lot of folks jumping into rails/grails without really thinking about what it really means and what the proper application of these tools might be. For example, I see folks deciding that rails/grails is the best development environment for them to build an application, then subsequently decide to write custom code for every thing. While rails/grails are still really good development frameworks no matter what, in many regards deciding to override their default behavior instead of understanding the patterns and embracing them is selling the idea short.

In my experience, I see many software applications as a similar pattern applied to itself. Rails/Grails (and thrust) are designed to exploit this and enable an exponential speedup in development time as well as a reduction in apparent complexity. I say apparent complexity because it still exists, but the framework should absorb the essential complexity (the part you cannot get rid of) and hide it from the developer in most circumstances.

Lets use an example: Suppose you are writing an application with a collection of screens that allow you to edit some information in tables in a relation database. Using any of these three frameworks, you can define a template that all these screens should use and automagically generate a good starting point for most of your screens (CRUD) + directory listing per entity.

If for some reason, you have no need for CRUD or a listing, or if you aren't using a database... you might be using the wrong tool for the job. That's not to say that these frameworks are not useful in this context, but you might not be getting the value that you COULD have from it.