mike.mainguy

Among many software architects and pundits, Message Queue solutions have a lot of press about being a highly scaleable solution in comparison with RPC based solutions. From what I can see, the biggest problem with most comparisons is that they start with the premise that one or the other of these two approaches is superior and then spend time trying to make a compelling argument why they are correct. I'm going to throw my hat in the ring on this issue and offer a high level guide for folks who don't have the time or energy to dig into queuing theory or debate with ivory tower architects about the issue. You'll note that scaleability is not even a factor. This is deliberate as scaleable and performant solutions can be built using either pattern. There is an interesting performance comparison that seems to indicate that the performance characteristics are very similar for both approaches. I WILL point out that simple http-based RPC solutions DO have fewer middleware re...

I recently went out to breakfast with my family at our local Denny's restaurant. We arrived around 9:15am and there was a pretty "interesting" line of folks waiting to be seated. In retrospect this should have been an indicator that something was wrong and things were going to be slow. We finally got seated and our server promptly brought menus and took our order, then we settled down and began waiting for our food. While waiting at out table, at least 6 other groups came in after us, were seated, ordered, got their food, and left. After about 30 minutes, the greeter actively started telling people that the food was going to take from 30-90 minutes to prepare and people stopped being seated. In addition, while the greeter was saying this to new customers, our server kept telling us the food would be out "in just a little bit". By 11:00am, I was pretty irritated because our breakfast had turned into lunch and all the other places to eat in the area were ...

Back in 2003 I wrote a quick application generator using turbine, java, and xml and published to sourceforge called thrust . It is extremely primitive by today's standards, but the important point is that is embraced the "convention over configuration" concept. I now see a lot of folks jumping into rails/grails without really thinking about what it really means and what the proper application of these tools might be. For example, I see folks deciding that rails/grails is the best development environment for them to build an application, then subsequently decide to write custom code for every thing. While rails/grails are still really good development frameworks no matter what, in many regards deciding to override their default behavior instead of understanding the patterns and embracing them is selling the idea short. In my experience, I see many software applications as a similar pattern applied to itself. Rails/Grails (and thrust) are designed to exploit this a...

I was reading up on the history of communication here and had to pause at how quickly things are changing. From the invention of written communication (3500BC) until the invention of the optical semaphore (1793), instantaneous and lossless long distance communication has been limited to how far you can shout/see. The rate of communication outside that range was limited to a max of about 200miles per day (speed of a horseback rider). This means, that for 5000 years it would take 1.5 days to send a message from Rome to Milan. Starting in 1793, this rate began to accelerate as a visual semaphore could drop that time significantly, but there was a huge amount of infrastructure to get this working. You needed towers, telescopes, and other things to make things work. In the space of 40 years, the electric telegraph greatly lowered the cost of long distance near light speed communication. For remainder of the 19th century, wired telegraph and eventually wireless telegraph lower the ...

Rational Application Developer Jabba the Hutt - BIG, SLOW, but also somehow powerful. You don't want to get on the wrong side of this IDE as you will be frozen in IBM consulting carbonite and NEVER get anything done. Netbeans Luke Skywalker- The hero who wins against all odds. However, In the software development universe, Luke actually turns to the dark side and joins Darth Vader. Visual Studio Emperor Palpatine - Having harnessed the dark side of the force, more powerful than you can ever imagine. Oracle JDeveloper Darth Vader - In the software world, after having turned Luke to the Dark side, Darth Ellison and Luke rule the Galaxy as father and son. TextMate Han Solo - Always seems to be there at the right time to help you out of a bind. No aspirations to rule the universe, but knows how to "get things done". Eclipse Battle Droids - There are millions of these, but they're all centrally controlled by someone whose motivations may be suspec...

Aside from supporting ruby, java, groovy, flex, and about a million other things, they actually have customer service. I don't mean "faceless mindless 3 levels of useless buearacracy" customer service, I mean "Holy crap, this guy WANTS to solve my problem" customer service. Recently I sent a note to intellij about an annoying, but not SUPER critical problem, to RAD or WSAD users (or just about any other software package), tell me the last time you sent an email and got this sort of help from a real person. Hello Michael, Please define "crashes". Serge Baranov JetBrains, Inc http://www.jetbrains.com "Develop with pleasure!" -----Original Message----- From: "Michael Mainguy" Sent: Tuesday, November 23, 2010, 7:29:23 PM To: feedback@jetbrains.com Subject: IntelliJ IDEA 'Feedback' Product: IntelliJ IDEA Build: IU-95.627 OS: Windows Name: Michael Mainguy Country: United States TimeZone: America/Chicago...

Web development is often hampered by the fact that there are a variety of web browser rendering engines as well as a variety of javascript interpreters. This means that a developer might have to recode the same web site 4-5 times to account for all the variations. When you couple that variation with the fact that new browsers are released and developed all the time, people started to realize that there needed to be a "one stop shop" to write your code and run it anywhere. I suppose someone at google started down the GWT path because "write once run anywhere" has been Java's watchword almost since it's inception and a software holy grail since the 1970s . A basic problem with GWT is that it tries to give a java api for building screens which is alien for almost all web designers and front end developers. This means that there is an additional translation from "designer" world into "developer" world. Recently had the pleasure to m...

OK, quickly, you've got a new app that has gone "off the charts", it's hosted on EC2 and you want to be able to scale in order to meet demand. What do you do? While this is a great situation, too often the answer technical people come up with is either: #1 (customer answer) We need to get someone else to build this us, our IT guys don't know what they're doing. #2a (developer answer) Rewrite the app in (erlang, scala, ruby, java, C#) because our code sucks and isn't scalable #2b (developer answer) Switch to (Oracle, DB2, MySql, MongoDB, Terracotta, Spring, EJB3) because (Oracle, DB2, MySql, MongoDB, Terracotta, Spring, EJB3) doesn't scale well #3a (infrastructure answer) We need to buy more EC2 instances and "scale out" #3b (intrastructure answer) We need to bring it in-house and we'll get the biggest baddest server you can buy #4 (architect answer) Where's the bottleneck? OK, I know #4 isn't really an answer, but it i...

Next time you find yourself in a pointless argument with someone who "knows" statistics, remember the bottom statistic. On the flipside, it took me at least 30 minutes of watching videos and reading explanations to figure out how it could even be possible to propel something directly downwind, faster than the wind, propelled only by the wind .

I've been investigating methods of storing content online and ran across an interesting offering from cloudant . They offer a 2gb couchdb database for free. For folks who don't know, couchdb is a json/RESTful distributed document database. If you're trying to manage online content for a web application it has some interesting advantages over the competition. The most interesting advantages to me are: Native RESTful javascript/JSON API. The database itself uses http as the communication protocol Inherent MVCC support. This means old versions of a document live after they've been updated Built-in searching and materialized views. I can define some metadata about my content and instantly retrieve it Some of the competition in this case would be: Mongodb Amazon S3 While it turns out that S3 and MongoHQ both have free offerings, the online console at cloudant is the most user-friendly (as of this second).

Next on my agenda for comparing these two frameworks is package (aka dependency management). Up until the release of Rails3, I would say grails was the hand's down clear cut winner in this regard. Grails was engineered from VERY early on with the idea of dependency management being core to the framework. IMHO, this further illustrates how grails advanced the state of the art by sanding off some of the rough edges off of rails. If I were in charge of an IT department, I still think grails has a bit of an edge from a management perspective, but it does lose out a little in the flexibility department. Ruby (via the gem mechnism) still suffers greatly from "gem hell" problems. Rails3 takes a step in the right direction by making bundler a core part of how applications are configured. Grails, on the other hand, is moving toward using maven as it's standard dependency management solution. In addition, grails has supported this for a number of years now and it works...

In my continuing comparison of ruby on rails to groovy and grails I've discovered another big difference. Grails has excellent support for job scheduling, whereas the existing rails plugins are confusingly complicated. In grails, to set up a job, install the quartz plugin grails install-plugin quartz grails create-job MyJob and edit the new class called MyJob class MyJob { static triggers = { simple name: 'mySimpleTrigger', startDelay: 60000, repeatInterval: 1000 } def group = "MyGroup" def execute(){ print "Job run!" } } Done, you now have a job running inside your application running at a repeating interval. The pluging supports cron-like syntax as well as custom triggers. Rails, on the other had (much like perl ;) has more than one way to do it. The biggest thing I notice is that most of the rails plugins either #1 require you to schedule a unix cron job example or #2 require you to run another ruby process to do the...

As technologist, I'm always keeping my eyes on the market. There's nothing worse from a marketability perspective than being the best chariot wheel repairman when the entire world has moved on to automobiles. If you're going to be in a niche, you better be in one that is HIGHLY lucrative. To this end, I took a look on indeed.com and tried to see how various programming languages stacked up as far as job postings. java, ruby, groovy, c#, php, c, erlang, vb, delphi Job Trends Java jobs - Ruby jobs - Groovy jobs - C# jobs - PHP jobs - C jobs - Erlang jobs - VB jobs - Delphi jobs Obviously this is not comprehensive, but it shows what I "kinda" already knew. C is king, with java taking a large secondary position and C# following up behind java. One thing to note is that that largest percentage shown on that chart is 4%. This roughly means that the market is SO fragmented that a large leader only captures 4%. For the COBOL folks clinging on fo...

Last month, I had a cloud server exploited and couldn't figure out how it happened. After a little investigation, I've got a good news bad news situation. The good news is that I DID manage to contact someone at rackspace who could help me out and they re-enabled my account. The bad news is that the server wasn't pretty. On the upside it must have been hacked by a script kiddie as they did NOT cover their tracks very well at all. On the downside, they did NOT appear to have used the single user account I created and somehow entered through either the rackspace admin network (SPOOKY, inside job?) or one of the default services installed with Ubuntu 10.04 LTS (still not good) From my root .bash_history, I noticed the following (the first few lines, may have been me): exit w w ...

I've been refactoring some tests and changed them from using a global set of users/roles defined as fixtures to instead be factories. OK, for java folks I'm going to give you the secret ruby decoder ring. Fixtures = predefined data that you create by manually seeding via seed.rb Factories = data generated via a factory method at runtime It's interesting that the ruby community has decided to overload the meaning of these terms to be very specific. I say this because in the "rest of the world" when dealing with testing, a test fixture is a much more generic concept. Typically it is the thing that sets up the test and tears down the test. Yes, often it creates data, but that is not necessarily it's job. Factories = This is a term that alludes to a well known and fundamental design pattern that can be used for a million different things and honestly has fallen out of vogue with java folks in favor of using dependency injection instead. It seems m...

As a person who has had the luxury to work in both ruby on rails and groovy grails, I've found a few differences that make their approach quite a bit different. #1 Groovy allows you to write java. While this isn't a huge deal, it can be both a positive and a negative. I've worked on teams where folks treat grails as a super simple java framework and never leverage any of groovy's dynamic goodness. While this isn't a huge problem, it does delay (or eliminate) to transition from J2EE xml configuration file hell into a more dynamic way of coding. #2 Ruby forces you to learn "the ruby way". For folks who are only used to java, seeing ruby code is like...seeing another language. Because of this, the idioms used in java are more quickly forgotten and you more quickly become a ruby native because you MUST. Only having worked with a few other people while they moved from java to ruby, I can only speak from my personal experience. I can say that ruby...

Modern web applications are all about user experience and a major factor in this is performance. A user interface that is laggy or gives the appearance of slowness will drive users away as quickly, if not more quickly, than an ugly and unintuitive one. This having been said, sometimes there are things that are just plain slow. Answering questions like "calculate the price for all 2 million products we sell in the north american market and present the top 10 with at least 50 in stock at a Distribution center within 50 miles" can often take some time. Couple these complex business rules with rich and powerful user interfaces and you have a potential for slowness to silently creep into your application. Digging through a few of the more modern javascript libraries, there are a number of strategies to combat this. We'll use the jquery datatable to illustrate some simple speedups that might apply. For our situation, let's pretend the above mentioned query takes 50...

I recently needed to stand up a DB2 server and was going to reach for my trusty rackspace account, but didn't feel like setting up DB2 for an experiment that would only last a few hours. Instead I turned to amazon. It turns out that amazon has preconfigured images for ubuntu/db2 that you can spin up almost instantaneously. In addition, their security model is a little more robust. Key things they do right from a security perspective (compared to rackspace). #1 They never send you a root password (via email or otherwise). You must generate a public/private key pair and download the key via https. Assuming you keep your secret key secure, there is minimal (if any) opportunity for someone to steal this key. Even if they hack your amazon account, I'm not sure they could get to your server immediately, even though they certainly could shut it down. #2 By default you are behind a firewall so that only a minimal set of tcp ports are even open. You need to actually take a...

I had an interesting discussion about locking in db2 a while back. It was interesting because it challenged some long held assumptions I had about db2 and how it handles locking. As usual, when I started digging deeper it turns out it is much more complicated that it would seem on the surface. First off, some background: I was having a conversation with a colleague about locking in various DBMS's and I made the statement that DB2 doesn't support MVCC. Thus, I contended, it is not possible for someone to read the previous version of a row that I have updated while I'm in a transaction that has altered it. At this point the fellow I was talking to looked at me as if I had just grown an arm out of my forehead. He stated (correctly, it turns out) that DB2 has supported this almost forever. I was, however, VERY confident that I was correct and subsequently dug up the documentation. Oddly enough, the documentation seemed to support the notion that I was mistaken (ga...

I've read with interest some documentation from Microsoft about how the HTTP 1.1 specification mandates some behavior. To Quote: WinInet limits connections to a single HTTP 1.0 server to four simultaneous connections. Connections to a single HTTP 1.1 server are limited to two simultaneous connections. The HTTP 1.1 specification (RFC2616) mandates the two-connection limit. This seems to be saying that browsers are only allowed (via some mythical mandate) to use two connections per server and any connections past two must block. After reading through the http 1.1 specification (again) I'm troubled that many folks have seriously misinterpreted this requirement. This is especially troubling because the manner in which RFCs are written is VERY explicit and it is (for me) really easy to understand the difference between a requirement and a recommendation. What is even more troubling is that people quote the microsoft reinterpretation of the specification as if it is a direct...

OK, so I've went round and round trying to figure out how my rackspace server was compromised and have come to the conclusion it was an inside job, but nobody's fessing up. I can see what sort of package was used to compromise my box and I may come back to trying to poison that package, but there isn't enough time in my life to continue to school folks who are hell bent on screwing with other people's property. Instead, I'll give folks a quick primer one how to have a little better security if you choose to use rackspace with ubuntu 10.04. #1 make sure your rackspace console password is secure... some basic rules: 10 chars, no dictionary words, upper and lower case letter, 2 number, at least one special char. #2 Once you get your root password emailed to you, log in via the secure console, and disable both network interfaces. I'm not sure what the 10.* interface is for, I'll figure that out later, but I'm assuming it's some sort of rackspace...

I finally had time to site down and actually analyze what happened to my box. It was certainly compromised by a script kiddie, but I'm not 100% sure if it was an inside job or not. In any event, I stored off the broken image and re-imaged the machine back to my "last known good" configuration. I subsequently added the account that I thought was used for the attack setting the password back to what it was when the machine was originally compromised, but limited login to the rssh shell. This has been running for almost a week with no problems now. I've had a couple of sniffs of folks trying to connect to my machine as root, but no solid hits. Some initial observations: First the default install of Ubuntu Server on rackspace's cloud accounts enables root login via ssh. This is very strange now that I think about it. Ubuntu, by default disallows this (for very good reasons ), and I think rackspace should seriously consider a change to their default build. ...

I've been using Rackspace cloud for testing some server builds and ESB solutions and recently ran into a "gotcha". First off, it looks like maybe the machine was compromised... I HOPE it was an inside job by one of my developer "friends" who happened to know the userid/password. If not, that means the default install of ubuntu 10.04, apache tomcat6, apache2.2, and servicemix is able to be compromised in less that 3 days when left out on the internet. In any event, that particular problem notwithstanding, I now have a different problem... That is, rackspace has suspended my account and I cannot access my server, nor create another one until Monday. Thank god I was only using this machine to test things, I can't image what I would have done if I was actually depending on it to be running. Another problem I'm finding is that I cannot find any reference on Rackspace's web site about acceptable use. They suspended the account for outbound ssh act...

Early Results Here are some early results: Port Huron, MI Rockford, IL These maps show the first page of garage sales on craigslist with about a 50% accuracy rate (meaning, only about 1/2 of time can I find an address). That having been said, it's still pretty impressive as manually entering these things into google maps is.... tedious. This process takes about 60 seconds per city using the script I've written. Back to Geocoding Note, geocoding is the process of attaching geographic coordinates to data. In my case I can find a "reasonable" address in about 1/2 of the entries. This means there is a string somewhere that looks like an address, notably, it has a few numbers, and then a couple of words. To get this data and geocode it, I wrote an awk script $1 ~ /\([0-9]+ .+\)/ {match($1,/([0-9]+ [^\)\(]+)\)/,out); printdata( "\""out[1]"\"")} $1 !~ /\([0-9]+ .+\)/ && $2 ~ /[0-9]+ [a-zA-Z]+ [a-zA-Z]+ /{match($2,/([0-9]...

The Backstory My wife is an avid garage sailer. She finds garage sales she thinks have promise, then cruises by them to see if there is anything of interest. She is so accomplished at this that she routinely turns a profit by snagging things that folks didn't realize had resale value, and flipping them at local consignment shops. While this doesn't pay the bills, it DOES provide enough extra cash to actually have her garage sailing at least pay for itself with a little left over. This is, however, not without it's share of problems: First, the postings online (or paper) for garage sales are scattered to the four winds. At this point, craigslist.org is the hands down winner for quality and quantity of posts. Local newspapers/classified also have a good quantity, but the few sites I found on google geared toward this are suffering terribly from a strategic chicken and egg problem. Second, while craigslist has a fairly high quality set of sales established, it h...

My internet service provider is a bit dicey and I occasionally need to be on the Internet even when they are figuring out how to reboot their remote router when they lose connectivity. So I thought I would just tether my smartphone (Droid ERIS) to my computer. After a bit of searching, I came up with a couple requirements. I didn't want to root my phone. While this is technically a cool thing to do, I just don't want to do that right now. I need to be able to connect natively to an Ubuntu linux machine. All my computers are currently running ubuntu and I didn't want to screw around with wine or a virtual machine. Enter easytether. In 5 minutes I had internet connectivity... here's what I did: downloaded easytether downloaded the ubuntu driver to the phone connected the phone to the computer (via usb) installed the .deb located in phone's download folder ran easytether on the phone ran "easytether enumerate" on my computer ran "sudo dhc...

Here's a copy of my verizon bill: My first reaction is "Hey, I don't owe anything"... However, from experience I think I DO, and I think it might be $127. Unfortunately, by putting a big $0 for my balance at the top, people generally are going to stop and simply assume they didn't owe anything. I've called Verizon about this a few times and 2/3 times the person I talk to ALSO thinks that I don't owe anything. I then need to talk them through my billing history for 15 minutes before they realize I DO in fact owe something. Usually this is after a supervisor gets involved and starts trying to explain complicated billing cycles and all sort of things that neither I nor the CSR actually care to know anything about. This is how NOT to design an online bill presentation screen, they've taken intimate knowledge about how their internal billing and accounting systems work and broadcast it all the way to the customer. In addition to frustration, this...

As a guy with a reputation of knowing something about computers, I often get hit up for tech advice from folks. Recently, a cousin of mine sent me a note asking about what sorts of things I post online and what I don't. Evidently he had been aware of a situation where kids used information from spokeo.com to commit crimes. Personally, I think this is pretty interesting and a really good question. I say this because I think a lot of non-tech folks have not yet made the transition from "off line" to "online". Many technical people have already had to deal with this (often years ago), but many younger folks and/or non-technical people are just beginning to understand the implications of being truly "online". For example, here is a post from 7 years ago by some buffoon (that's me) who decided a to post an off topic friday afternoon 833r discussion. This will likely be available for a very very long time. From my perspective, this is a prett...

I have a situation where I need to be able to have sub groupings of users in Active directory to manage who can see particular pieces of information. It turns out this is easy, but unintuitive. An important detail is to realize that groups can be put inside into other groups and you can use the "member" and "memberOf" attributes to determine who is in which group. So if you have an OU in Active directory called "OU=web,DC=mainguy,DC=org" and you create a group with a name of "CN=Germany National Sales,OU=web,DC=mainguy,DC=org". From here in Active directory you create any number of subgroups and put them in the parent group (under the same OU in our example, but that's not necessary). At this point, you can dump users into any of the groups and you can get segregate users into nested structures. With a little creativity you can use recursion to have deeper nesting (not necessarily a good thing) as well as a "deny/allow" capab...

OK, I realize from a previous post that it in a previous post it may seem like I think that windows is completely non-functional in a cloud environment. Let me back up a little... From my perspective, running production servers on virtual machines spun up on demand is not useful (yet) with the windows operating system. The windows OS strategy is just not responsive enough to this sort of business requirement (yet). That having been said, I just finished spinning up a windows 2003 server instance on rackspace... Why? I need to investigate some Active Directory problems we're trying to solve at work right now. I spent 4 hours downloading the massive DVD install image (and a bunch of crazy MS registration stuff) for a 10 day TRIAL version of the OS. I then went over to my rackspace account (because of a different problem) and realized they could set up the server I needed. I clicked "create server" and they set up a virtual server in 15 minutes. Yes, it costs ...

I happened to notice that mygopher.com doesn't seem to work properly with firefox 3.6 OR google chrome... That's about 25-50% of "normal" web traffic. Note: if the techie people show you log files that indicate firefox is really only 5% (or some other low number).... ask them "how could ANYONE with firefox possibly be using the site if it doesn't actually WORK with firefox?". I used to be a little self conscious about posting gripes about browser compatibility, but now that I see the real numbers from a number of sites that get millions of hits, I'm fairly confident that firefox (and even crome) are actually pretty important at this point. Right now I support a couple of non-technical sites that get millions of hits per month and IE gets around 70% firefox gets around 20% and chrome gets around 5% (the rest is a mixed bag). I'd recommend letting the tech folks take about a week and make the site at least work with these browsers. In al...