Thursday, September 25, 2014

Why OSX sucks and you should use ubuntu instead

OK, I confess, I use OSX almost exclusively and have for a number of years now. I DO have a number of Ubuntu machines, VMs, and servers in stable, but my goto device is a macbook pro (actually two of them, one for work, one for fun). I love the hardware, but the OS and specifically it's lack of a software package management tool has just a level of suckyness that irritates me.

Now, don't get me wrong, OSX suckyness is nothing compared to windows, but it seems to be frozen in 2004 and is not moving forward at a pace I think is acceptable considering the huge advances Ubuntu has made in a very short time frame. In the same vein, the UI for OSX is awesomely polished and user friendly, but there are some major pain points I can't seem to get past.

My Points

Ubuntu, being a Debian variant has an awesome software package management system. More importantly, just about anything you could ever want is ALREADY THERE in some shape or form. OSX has homebrew and macports...which both suck and are just plain confusing. Why in the world there is a need to do a recompile on a platform as tightly controlled as OSX when Ubuntu can deploy binary packages is a complete mystery to me.

This having been said

Apple is a hardware and user experience company, not a software company. Your hardware awesomely rawks, keep it up. Your software is pretty darn good, but you need partner with canonical and/or an open source company to get a decent package management solution (or just fork Debian...or just partner with canonical). Your development tools are horrific. Please contact a professional developer who also does open source, not a sycophantic Apple Fanboi to help fix the problem.

Monday, August 18, 2014

It's not NoSQL versus RDBMS, it's ACID + foreign keys versus eventual consistency

The Background

Coming from a diverse background and having dealt with a number of distributed systems, I routinely find myself in a situation where I need to explain why foreign keys managed by an acid compliant RDBMS (no matter how expensive or awesome), lead to a scaleability problem that can be extremely cost prohibitive to solve. I also want to clarify an important point before I begin, scaleability doesn't equate to a binary yes or no answer, scaleability should always be expressed as an cost per unit of scale and I'll illustrate why.

Let's use a simplified model of a common web architecture.

In this model, work is divided between application servers (computation) and database servers (storage). If we assume that a foreign key requires validation at the storage level, no matter how scaleable our application layer is, we're going to run into a storage scaling problem. Note: Oracle RAC is this model...at the end of the day, no matter how many RAC nodes you add, you're generally only scaling computation power, not storage.

To circumvent this problem, the logical step is to also distribute the storage. In this case, the model changes slightly and it begins to look something like this.

In this model, one used by distributed database solutions, (including high end acid compliant databases such as Oracle RAC or Exadata or IBM purescale), a information storage is distributed among nodes responsible for storage and the nodes don't share a disk. In the database scaling community, this is a "shared nothing" architecture. To illustrate this a little further, the way most distributed database work in a shared nothing architecture is one of two ways, for each piece of data they either:

  • Hash the key and use that hash to lookup the node with the data
  • Use master nodes to maintain the node to data association

So, problem solved right? In theory, especially if I'm using a very fast/efficient hashing method, this should scale very well by simply adding more nodes at the appropriate layer.

The Problem

The problem has to do with foreign keys, ACID compliance, and the overhead they incur. Ironically, this overhead actually has a potentially serious negative impact on scaleability. Moreover, our reliance on this model and it's level abstraction, often blinds us to bottlenecks and leads to mysterious phantom slowdowns and inconsistent performance.

Let's first recap a couple of things (a more detailed background can be found here for those that care to read further.

  • A foreign key is a relation in one table to a key in another table the MUST exist for an update or insert to be successful (it's a little more complicated than that, but we'll keep it simple)
  • ACID compliance refers to a set of rules about what a transaction means, but in our context, it means that for update A, I must look up information B

Here's the rub, even with a perfectly partitioned shared nothing architecture, if we need to maintain ACID compliance with foreign keys, we run into a particular problem. If the Key for update A is on one node, and the Key for update B is on a different node... we require a lookup across nodes of the cluster. The only way to avoid this problem... is to drop the foreign key and/or relax your ACID compliance. It's true that perfect forward knowledge might allow us to design the data storage in such a way that this is not really a problem, but reality is otherwise.

So, at the end of the day, when folks are throwing their hats into the ring about how NoSQL is better than RDBMS, they're really saying they want to use databases that are either:

  • ACID compliant and they'll eschew foreign keys
  • Not ACID compliant

And I think we can see that, from a scaleability perspective, there are very good reasons to do this.

Friday, August 15, 2014

Things to remember about information security

As more businesses look to cloud application providers for solution, the need for developers to understand secure coding practices is becoming much more important. Gone are the days when a developer would write an application that only ran in a secure environment and now it is possible for applications to be moved to locations where previously well managed security gaps now are exposed to the internet at large. Developers now more than ever need to understand basic security principles and follow practices to keep their applications and data safe from attackers.

To make things more secure, a developer needs to first understand and believe the following statements:

  • You don't know how to do it properly
  • Nothing is completely secure
  • Obscurity doesn't equal security
  • Security is a continuum

You don't know how to do it properly

If I had a nickel for every developer who though they invented the newest, greatest, cleverest encryption/hashing routine, I'd be a millionaire. Trust me, if you aren't working for the NSA or doing a doctorate on the subject, there are thousands of people who can defeat your clever approach...worse yet, even if you ARE in the aforementioned groups there are still SOME folks who can defeat your approach. Which means:

Nothing is completely secure

The only way to completely secure a system or data is to completely destroy it. This is a mathematical fact, don't argue, just trust me on this. If ONE person can access the information, someone else can. MAYBE if it's in your head and your head alone it is pretty secure, but there are ways of getting that information too...some of which can be unpleasant. So these two things having been said, I want to add the clarifying statement that:

Obscurity doesn't equal security

As someone who has witnessed back doors get exploited numerous times, thinking you can just "hide the key under the rock" and hope for the best is not a sound policy. Don't get me wrong, making targets less obvious is great... please do it... but be wary of relying on this as your sole security measure, it will be discovered. Which leads to my final point:

Security is a continuum

Remember how security isn't absolute? Well this is the reassertion of that statement. When having discussions, the question isn't "is it secure (yes/no)?" it should be "is it secure enough (yes/no)?" and "what are our threat vectors?". Subtly changing the question from being absolutely yes or no can open up a discussion and let you objectively begin to measure your risk.