The Magnitude That Matters

I quite enjoyed reading Joe Duffy’s piece: The ‘premature optimization is evil’ myth, highlighting just how misused that Knuth quote often is. There’s a lot of good stuff that should always be considered, especially around picking the right data structure – mostly to improve maintenance more so than performance.

The most important point I find in the piece is to understand the order of magnitude that matters:

First and foremost, you really ought to understand what order of magnitude matters for each line of code you write.

In other words, you need to have a budget; what can you afford, and where can you afford it? The answer here changes dramatically depending on whether you’re writing a device driver, reusable framework library, UI control, highly-connected network application, installation script, etc. No single answer fits all.

I am personally used to writing code where 100 CPU cycles matters. So invoking a function that acquires a lock by way of a shared-memory interlocked instruction that may take 100 cycles is something I am apt to think hard about; even more worrisome is if that acquisition could block waiting for 100,000 cycles. Indeed this situation could become disastrous under load. As you can tell, I write a lot of systems code. If you’re working on a network-intensive application, on the other hand, most of the code you write is going to be impervious to 100 cycle blips, and more sensitive to efficient network utilization, scalability, and end-to-end performance. And if you’re writing a little one-time script, or some testing or debugging program, you may get away with ignoring performance altogether, even multi-million cycle network round-trips.

This concept of order of magnitude is why premature optimization turns out to be evil so much of the time – there aren’t that many areas in software development that require such serious, CPU-bound, performance. Once you understand what types of performance you need, where you need them and what will stop you achieving it, you’re no longer doing premature optimization. Any optimization you do at that point is warranted and well informed.

I find a lot of parallels in many of the language brain teasers and trivia questions. They can be interesting, but what’s important to know about them isn’t the right answer, just knowing when to look it up is what matters. For example, Java programmers don’t need to know that an int is 32-bit, they really just need to know that an int isn’t limitless and it goes from Integer.MIN_VALUE to Integer.MAX_VALUE.

Know what matters, focus on that and do the simplest thing that can possibly work.