Monday, November 29, 2010

The internet is causing the world to shrink

I was reading up on the history of communication here and had to pause at how quickly things are changing. From the invention of written communication (3500BC) until the invention of the optical semaphore (1793), instantaneous and lossless long distance communication has been limited to how far you can shout/see. The rate of communication outside that range was limited to a max of about 200miles per day (speed of a horseback rider).

This means, that for 5000 years it would take 1.5 days to send a message from Rome to Milan. Starting in 1793, this rate began to accelerate as a visual semaphore could drop that time significantly, but there was a huge amount of infrastructure to get this working. You needed towers, telescopes, and other things to make things work.

In the space of 40 years, the electric telegraph greatly lowered the cost of long distance near light speed communication. For remainder of the 19th century, wired telegraph and eventually wireless telegraph lower the cost of high speed and distant communication. While the actual transport happened at or near the speed of light, the encoding cost slowed things down. Wireless telegraph was FAST, but nobody really had that in their house so they had to walk to the telegraph office and have a specialist encode the message.

In the 20th century, the advent of in-home telephone and television enabled nearly instantaneous worldwide communication. In the 15+ years since WWW began, the actual speed of communication has somewhat maxed out. You can't get much faster than light and short of passing messages through the earth instead of around, you'll have to accept a few hundred milliseconds of latency. Effectively, the entire world is living together. What would have taken days, months, or even years to communication even 100 years ago, can now happen instantaneously.

This means that the communication barriers caused by geographic dispersion are now effectively gone. If you can get an Internet connection, you are within shouting distance of the entire world.

Sunday, November 28, 2010

If IDE's Were Star Wars Characters

Rational Application Developer

Jabba the Hutt - BIG, SLOW, but also somehow powerful. You don't want to get on the wrong side of this IDE as you will be frozen in IBM consulting carbonite and NEVER get anything done.


Luke Skywalker- The hero who wins against all odds. However, In the software development universe, Luke actually turns to the dark side and joins Darth Vader.

Visual Studio

Emperor Palpatine - Having harnessed the dark side of the force, more powerful than you can ever imagine.

Oracle JDeveloper

Darth Vader - In the software world, after having turned Luke to the Dark side, Darth Ellison and Luke rule the Galaxy as father and son.


Han Solo - Always seems to be there at the right time to help you out of a bind. No aspirations to rule the universe, but knows how to "get things done".


Battle Droids - There are millions of these, but they're all centrally controlled by someone whose motivations may be suspect. In the software development world, battle droids are actually secretly controlled by Samuel the Hutt.


Yoda - In the software universe, much like in the Star Wars universe, he may be a powerful Jedi, but he's dead.


Obi wan Kenobi - Strong in the force, but for some reason he lives in obscurity on a desert planet. He and Joda live on in some mysterious way that we don't quite understand.


An Ewok - Cute, might be actually useful, but do you seriously want to try and rule the universe as an Ewok?


R2D2 - Very useful, behind the scenes kinda hero. Never going to rule the universe, but someone you certainly want to have around all times of possible.

Flash Builder

Padme Amidala - Not sure what to say here.


This is my list so far and I realize it isn't really complete. Comment and I'll try to update with your ideas...

Saturday, November 27, 2010

Why you should purchase Intellij.

Aside from supporting ruby, java, groovy, flex, and about a million other things, they actually have customer service. I don't mean "faceless mindless 3 levels of useless buearacracy" customer service, I mean "Holy crap, this guy WANTS to solve my problem" customer service.

Recently I sent a note to intellij about an annoying, but not SUPER critical problem, to RAD or WSAD users (or just about any other software package), tell me the last time you sent an email and got this sort of help from a real person.

Hello Michael,

Please define "crashes".

Serge Baranov
JetBrains, Inc
"Develop with pleasure!"

-----Original Message-----
From: "Michael Mainguy"
Sent: Tuesday, November 23, 2010, 7:29:23 PM
Subject: IntelliJ IDEA 'Feedback'

Product: IntelliJ IDEA
Build: IU-95.627
OS: Windows

Name: Michael Mainguy
Country: United States
TimeZone: America/Chicago
Evaluator: true

> Are you generally satisfied with IntelliJ IDEA? How do you rate the product?
Generally yes, very good

> What features appear most useful? Are there any problems?
Yes, GWT 1.4 project crashes when I try to change project structure.

> Are there any features you'd like to have but did not find in IntelliJ IDEA?
Not yet.


Note, he emailed me at 10:31 the next morning when I sent my note at 7:30 pm the night before...

So I reply to him at 12:55

By crash, I mean I get a little popup with a stack trace that I can't seem to copy. The IDE itself remains running and seems to not have any other problems.

Maybe crash is not the right word... but I'm the user so get to make vague overstatements right? ;)

If I could figure out how to get a copy of the stack trace I'd report a bug, but I'm busy fumbling along without the debugger for the time being.
- Show quoted text -

I hit send, then turn to ask a coworker a question, when I turn back:

Hello Mike,

There should be blame button which will post stacktrace to us. Also
is should be possible to copy from this dialog by selecting text and
pressing ctrl+c.

Exception is also logged in
USERPROFILE\.IntelliJIdeaXX\system\log\idea.log (where XX is IDEA
version) or ~/Library/Logs/IntelliJIdea90/idea.log if you are on Mac.

Serge Baranov
JetBrains, Inc
"Develop with pleasure!"

I felt a little like Serge is the sneaky butler from Mr Deeds or at a minimum, a Jimmy Johns Delivery guy. In the next email exchange, he gave me some instructions on how to fix the problem (delete broken findbugs plugin metadata) and my issue was resolved.

The sad thing is that my expectations were so low, that actual customer service is seen as "over the top". I issue an open challenge to any owner of RAD to share a similar experience with customer support form IBM for their tool that costs over 10x more than mine. I won't even engage in a debate about the features of RAD versus IntelliJ since those are pretty subjective. After all, I've only met a handful of people who ever actually enjoyed working with RAD and most of them only used 10-15% of the "features" in the tool.

While I still use eclipse and actually like STS, Intellij Idea is so much more convenient and their customer service is so spectacular that I highly recommend it for anyone doing development in the java/ruby space. Just so we're clear, here's what intellij idea supports out of the box:Java, JavaScript, HTML/XHTML/CSS,
ActionScript/Flex/AIR, XML/XSL, Ruby/JRuby, Groovy, SQL, FreeMarker/Velocity, PHP.

In addition, it supports a number of other languages with plugins.

Thursday, November 18, 2010

GWT is not a substitute for a web developer

Web development is often hampered by the fact that there are a variety of web browser rendering engines as well as a variety of javascript interpreters. This means that a developer might have to recode the same web site 4-5 times to account for all the variations. When you couple that variation with the fact that new browsers are released and developed all the time, people started to realize that there needed to be a "one stop shop" to write your code and run it anywhere.

I suppose someone at google started down the GWT path because "write once run anywhere" has been Java's watchword almost since it's inception and a software holy grail since the 1970s. A basic problem with GWT is that it tries to give a java api for building screens which is alien for almost all web designers and front end developers. This means that there is an additional translation from "designer" world into "developer" world.

Recently had the pleasure to muddle through some GWT code and I don't really like what I see. For folks who don't know, GWT (I love TLAs) is tool that enables developing web front ends in pure java. This is done by running a custom compiler against java source code that outputs javascript. The cross platform capabilities get added in by generating 5 different javascript files, one for each browser platform you are targeting.

Because of this, some shops appear to try and co opt java developers and encourage them to build web front ends and this is NOT a good idea. Does this mean a good web designer or user interaction person who knows java (or visa versa) won't be able to make great things with it? Of course not.... But it does NOT mean that a band of capable back-end java developers will be able to suddenly developed a rock star web 4.0 interactive application because "they know java" and "we're using GWT".

GWT does NOT give java developers magical design abilities, nor does it somehow make a java backend developer able to write wicked fast javascript. It DOES enable a person who understands javascript and web design as WELL as java to build cross platform applications MUCH faster than they if they tried to do it themselves.

In conclusion, GWT certainly has it's place in a capable software developer's toolbox as a mechanism to build screens, but as a replacement for front end developers or some sort of attempt at replacing knowledge of how browsers work, it is a pretty bad idea.

Tuesday, November 16, 2010

Architecture and Scaling Cloud Applications

OK, quickly, you've got a new app that has gone "off the charts", it's hosted on EC2 and you want to be able to scale in order to meet demand.

What do you do?

While this is a great situation, too often the answer technical people come up with is either:
#1 (customer answer) We need to get someone else to build this us, our IT guys don't know what they're doing.
#2a (developer answer) Rewrite the app in (erlang, scala, ruby, java, C#) because our code sucks and isn't scalable
#2b (developer answer) Switch to (Oracle, DB2, MySql, MongoDB, Terracotta, Spring, EJB3) because (Oracle, DB2, MySql, MongoDB, Terracotta, Spring, EJB3) doesn't scale well
#3a (infrastructure answer) We need to buy more EC2 instances and "scale out"
#3b (intrastructure answer) We need to bring it in-house and we'll get the biggest baddest server you can buy
#4 (architect answer) Where's the bottleneck?

OK, I know #4 isn't really an answer, but it illustrates the problem. The architect's job is to fully understand the problem and help guide discussion about what possible solutions are. I've met a lot of architects and usually can tell what their background is after a discussion like this. The well rounded folks will typically ask a lot of questions before jumping to a conclusion about the best answer. The folks who grew up in the business will choose #1, Developers #2, Infrastructure #3, and rarely you'll get a #4.

That having been said, the best answer is probably to get ahead of the problem and build scalability into your design. For dynamic web applications this means you should adopt the following positions:

#1 My application might be hosted on a laptop with an in memory database, or it might be hosted on 52 app servers with 12 database instances - my design shouldn't require large changes to accomodate either.
#2 My URL scheme matters (even down to DNS) and should reflect unique identities of resources that my application deals with (see REST AND use proper http verbs where applicable.
#3 Don't panic

So what does this really mean?

For one, the days of writing a J2EE app, and/or SOAP as the best way to do things are past. While J2EE can scale up easily and theoretically even scale out, it's just too dang expensive and difficult to get that to work well (unless you like throwing money at problems). Why?

First off, most J2EE containers (and applications) are designed around the idea that you have a big chunk of shared memory and multiple processors available. As an example, many of the shared resources rely on thread pooling. This has some serious drawbacks when you start getting into extremely high load. All those thread pools need overhead to manage each other's state and that management affects your entire application as all those CPUs start to melt down doing thread management activities.

Second, SOAP is basically a more web centric platform independent replacement for CORBA... Because of it's design, it requires a bit more work than simple URL parsing to be able to scale it. Add to this the problem of xml marshalling and you'll quickly devlolve into a big mess of performance and scalability problems.

These problems aren't the only ones, nor are they limited to J2EE and web services (.Net suffers from similar problems), but they are indicative of a bigger problem brewing in web application development and architecture. To get ahead of this, start thinking about what is is REALLY NECESSARY to run your application. It may be surprising how much extra "stuff" you've added in "just in case" that is not only hampering development tempo, but also potentially slowing you down at runtime.

Tuesday, November 9, 2010

Just enough math to be dangerous

Next time you find yourself in a pointless argument with someone who "knows" statistics, remember the bottom statistic.

On the flipside, it took me at least 30 minutes of watching videos and reading explanations to figure out how it could even be possible to propel something directly downwind, faster than the wind, propelled only by the wind.

Wednesday, November 3, 2010

Cloudant couchdb is free

I've been investigating methods of storing content online and ran across an interesting offering from cloudant. They offer a 2gb couchdb database for free. For folks who don't know, couchdb is a json/RESTful distributed document database. If you're trying to manage online content for a web application it has some interesting advantages over the competition.

The most interesting advantages to me are:
  • Native RESTful javascript/JSON API. The database itself uses http as the communication protocol
  • Inherent MVCC support. This means old versions of a document live after they've been updated
  • Built-in searching and materialized views. I can define some metadata about my content and instantly retrieve it

Some of the competition in this case would be:

While it turns out that S3 and MongoHQ both have free offerings, the online console at cloudant is the most user-friendly (as of this second).