If you talk to Rubylosophers, you’ll get some hip and hippie discussions of how languages should be designed for harmony and balance, to make programmers happy. When faced with a Rubygmatic programmer, words like ”productivity” and ”rapid development” will be heard. However, those seem almost magical, and are pretty hard to base a hard decision on unless you’ve experienced them yourself.

In this post, I’ll explain a simple model that I’ve come up with over the years, as I experienced a fair (though by no means vast) variety of languages. Please note that it’s only a model, and it’s not meant to explain everything under the sun. It’s simply a metaphor that I use when describing why language X is better than language Y to someone who doesn’t get programming. If you push and pull at it enough, it will crumble. But before it does, it can still be used to derive some interesting insights. I hope it is useful to others too.

Since I’ve come up with a number of interesting insights from this model, I’m going to split this article into multiple parts, probably about 3.

The basic model

The key piece of thinking behind this model is the idea that as a program ages and more functionality is pinned onto it, it develops something I like to call “viscosity”.

What is programming viscosity? It’s the almost imperceptible (at first) slowing down of the speed at which change can occur, due to the risk of causing unforeseen bugs. It’s not a new concept, nor is it only applicable to software. In ”Developing products in half the time”, which has nothing to do with software, the authors argue that each feature increases the complexity of the system in an exponential manner (because each new feature adds N possible interactions to the product, where N is the number of existing features). That is largely true for software as well, but in a much more pernicious way.

Software products are inherently vulnerable to scope creep, much more so than physical products, because “customers” (be they clients, or business users within your company), have an intrinsic understanding that adding more buttons to a physical object makes it more complex, but they don’t have such a direct understanding of why the same is true for software (a whole article could be written about why that is).

Now, viscosity in software products is never actually infinite. It’s always possible to make changes. However, the viscosity may well become so large that each change ends up taking a huge amount of time. When that happens, change freezes because it’s too expensive. Not only that, but as the complexity increases, it becomes more difficult for programmers to deal with it. Either they get irritated and bored by it, in which case they’ll figure out a way to get out of your project and move elsewhere, or they simply can’t cope with it anymore, and then you get a situation like that described in this article, where you need a vastly better programmer to come in and sort out the mess (effectively rewriting half or all of the system in the process). If you don’t have access to one of those great programmers when that happens, you’re stuffed. If you do, you may still be stuffed, but at least you have a chance.

This state of high viscosity is, in my experience, inevitable. No matter how smart your programmers and how fantastic your language, they’ll eventually reach it, unless your project is really trivial (e.g. an online company brochure). All non-trivial projects accumulate features over time and get harder and harder to work with. The question is, can you push this point back until you can actually afford a major rewrite? If you pick the wrong language, what’ll happen is you’ll reach that point too quickly, before you’ve shipped your product and started taking in cash. And then your business will sink. Even if you have vast amounts of money available to patch things up, you want to push back that point as far as possible, because a Big Rewrite is always a costly and risky manoeuvre, with no guarantees of success.

Without further ado, here’s the basic graph showing what I’ve been talking about.

There’s only one new element that I haven’t talked about in this graph, and that’s the Boredom Limit. What this limit means is quite simple: beyond this point, your programmers will lose interest. They’ll still work, maybe even work hard, but it’ll be something other than the love of coding something cool that keeps them at it (good examples might include: financial bonuses, personal commitment, enthusiasm for the business idea behind the project). Even if they continue to work on the project, there’ll be a nagging voice inside telling them they’d rather be doing something else, and that will start to take its toll as time passes. This extra cost won’t kill your project, but you should be aware of it so that you can ensure those additional incentives are in place - otherwise your programmers will just leave.

So, let’s draw a first insight from this graph.

Insight 1: Comparing languages

Let’s represent more than one language on this graph. Each person might have their own opinion about languages. The comparison below represents my experience with 3 fairly important technologies used to develop web applications: Java, PHP and Ruby on Rails.

There are basically two factors at play here: the initial overhead, which increases the viscosity right from the get-go; and what I’ll call the “viscosity function”, which is the speed at which viscosity increases with time.

A framework like PHP has very little overhead at first, which is extremely good for very small projects that have extremely little chance of growing beyond that simple stage. For instance, a very lightly dynamic website with some simple forms would be very quick to develop in PHP, and slower to develop in most other frameworks. However, as the project grows, the viscosity of a PHP application quickly becomes unmanageable. Since all medium projects have a tendency to accrue extra features and become larger, I would not recommend PHP for anything that is likely to be more than a simple site with a handful of forms and dynamic elements.

At the other extreme, a language like Java, with a rock-solid framework like J2EE, has enormous up-front overheads. It takes about a week just to get your development environment wired up so you can get started on your development! By that time, the PHP project should already be finished and shipped. In From Java to Ruby, Bruce Tate quotes some engineering managers who claim that it takes at least 2 years to earn the adjective “experienced” in the Java world. And, from my personal experience, that’s true. There are many, many bits and pieces to learn in a framework like J2EE, and it takes an enormous span of time to acquire them all, and just as long to wield them effectively. On the good side, however, J2EE scales admirably well into the future, so long as it’s been reasonably well architected. Your development might be a pain, lumbering along like a snail right from the beginning, but you can rest fairly well assured that it will be a looong time before it slows down much more than that.

Finally, in the middle, a framework like Ruby on Rails provides a much, much better viscosity function than PHP, but with a much, much lower initial overhead than J2EE. Rails supports most of the useful J2EE patterns right out of the box, so that reduces the viscosity function quite a bit. It’s very quick to learn and be productive in (I started doing cool things with it practically from day one). So the initial overhead is low.

Placing these languages like that is only my opinion, of course. Feel free to disagree. However, this is the way I’d place them based on my experience of them.

That’s enough for this first article. I’m very keen to hear what others think about this model, so please leave some comments if you have some suggestions. Part 2 and 3 of this article will cover how this model can be extended to explain the effects of good and bad architecture, of development methodologies and of programmer quality (and probably a couple more things I haven’t thought of yet).

Thanks for reading.

Step 1: First, it was simple

Then("user $email should receive an email with his confirmation link") do |email|
  mail = ActionMailer::Base.deliveries.last
  user = User.find_by_email(email)
  mail.body.match(/#{user.confirmation_code}/) && mail.body.match(/#{user.id}/)
end

This is my simple matcher that I started with… The world was simple, and this worked fine. But then, something happened. The world started sending more than a single email in one Story, and so the need for matching multiple emails was born.

Step 2: Then, it was easy, but messy

Then("user $email should receive an email with his confirmation link") do |email|
  matches = 0
  ActionMailer::Base.deliveries.each do |mail|
    user = User.find_by_email(email)
    matches += 1 if (mail.body.match(/#{user.confirmation_code}/) && mail.body.match(/#{user.id}/))
  end
  matches.should == 1
end

This works. It works well. It works fine. But, it’s ugly. It’s not readable. It’s not RSpec-like, and it stands out like a sore thumb in the middle of my step matchers. I have Ruby to thank for the fact that fugly hacks like this stand out and beg to be rewritten in a neater way. Other languages, such as PHP, do not have this quality, and so hacks remain untouched for ages and ages.

I could not let things rest in this state. Ugliness like that must be removed. Trying to match any one of an array of items is one of those things that are likely to be repeated a few times, so I decided to extract that out. This is where another one of Ruby’s qualities came forth, and made it child’s play to extract this concept and stick it where it belongs - in Enumerable (but, I hasten to add, only when running specs…).

Step 3: Then, it was simple again

Enter my latest addition to /stories/helper.rb:

ENV["RAILS_ENV"] = "test"
require File.expand_path(File.dirname(__FILE__) + "/../config/environment")
require 'spec/rails/story_adapter'

module Enumerable
  # my_array.should_match_at_least_one { |item| item.match(/abc/) && item.match(/def/) }
  def at_least_one_should
    matches = 0
    self.each do |item|
      matches +=1 if yield item
    end
    matches.should >= 1
  end
end

Which means that the step matcher can now been reduced to the reasonably elegant and expressive:

Then("user $email should receive an email with his confirmation link") do |email|
  user = User.find_by_email(email)
  ActionMailer::Base.deliveries.at_least_one_should do |mail|
    mail.body.match(/#{user.confirmation_code}/) && mail.body.match(/#{user.id}/)
  end
end

You might be tempted to point out that I could probably use String#include? instead of String#match, since I do not care about the actual matching, only about the presence… and you’d be right. But, like an imperfectly perfect persian rug, I’m happy with my code like this.

In his article The 18 mistakes that kill startups, Paul Graham makes the following point:

“… what killed most of the startups in the e-commerce business back in the 90s, it was bad programmers. A lot of those companies were started by business guys who thought the way startups worked was that you had some clever idea and then hired programmers to implement it. That’s actually much harder than it sounds—almost impossibly hard in fact—because business guys can’t tell which are the good programmers. They don’t even get a shot at the best ones, because no one really good wants a job implementing the vision of a business guy.

In practice what happens is that the business guys choose people they think are good programmers (it says here on his resume that he’s a Microsoft Certified Developer) but who aren’t. Then they’re mystified to find that their startup lumbers along like a World War II bomber while their competitors scream past like jet fighters. This kind of startup is in the same position as a big company, but without the advantages.

So how do you pick good programmers if you’re not a programmer? I don’t think there’s an answer. I was about to say you’d have to find a good programmer to help you hire people. But if you can’t recognize good programmers, how would you even do that?”

I disagree with Mr Graham on this one. I think there are a number of very strong indicators of a “good programmer” (and, conversely, strong indicators of a “not-so-good programmer”) that even a business guy can recognise. I’ll summarise some key indicators and counter-indicators in a list at the end of the article.

#1 : Passion

In my corporate experience, I met a kind of technical guy I’d never met before: the career programmer. This is a person who’s doing IT because they think it’s a good career. They don’t do any programming in their spare time. They’re shocked when they find out I have a LAN and 3 computers at home. They just do it at work. They don’t learn new stuff unless sent on a training program (or motivated by the need to get a job that requires that technology). They do “programming” as a day job. They don’t really want to talk about it outside of work. When they do, they talk with a distinctive lack of enthusiasm. Basically, they lack passion.

I believe that good developers are always passionate about programming. Good developers would do some programming even if they weren’t being paid for it. Good programmers will have a tendency to talk your ear off about some technical detail of what they’re working on (but while clearly believing, sincerely, that what they’re talking about is really worth talking about). Some people might see that as maladapted social skills (which it is), but if you want to recognise a good developer, this passion for what they’re doing at the expense of social smoothness is a very strong indicator. Can you get this guy to excitedly chat up a technology that he’s using, for a whole half hour, without losing steam? Then you might be onto a winner.

#2 : Self-teaching and love of learning

Programming is the ultimate moving target. Not a year goes by without some new technology robbing an old, established standard blind and changing half the development universe. This is not to say that all good programmers pick up these changes and ride the bleeding edge. However, there’s a class of programmers that will never, ever pick up a new technology unless forced to, because they don’t like learning new stuff. These programmers will typically have learnt programming at university, and expect to get by on whatever skills they picked up there, plus whatever courses their company is willing to send them on.

If you’re thinking of hiring someone as a programmer, and he ever utters the words “I can work with that, just send me on a training course for a week and I’ll be good at it”, don’t hire that guy. A good programmer doesn’t need a training course to learn a new technology. In fact, the great programmer will be the one talking your ear off about a new technology that you haven’t even heard of, explaining to you why you must use it in your business, even if none of your staff knows how to use it. Even if it’s a technology he doesn’t know how to use yet.

#3 : Intelligence

Some business people assume that lack of social tact and lack of intelligence are the same. Actually, intelligence has several facets, and emotional/social intelligence is only one of them. Good programmers aren’t dumb. Ever. In fact, good programmers are usually amongst the smartest people you know. Many of them will actually have pretty good social skills too. The cliché of the programmer who’s incapable of having a conversation is just that - a cliché. I’ve been to a few meetings of the London Ruby User Group and I can say that with only a very few exceptions, most people there are smart, talkative, sociable, have varied interests, etc. You wouldn’t look at them chattering away in the pub and think “what a bunch of geeks!” - at least until you approach a group and realise they’re talking about the best way to design a RESTful application with a heavy UI frontend.

This doesn’t mean that they’ll all feel comfortable in every social context. But it does mean that if the context is comfortable and non-threatening enough, you’ll be able to have as great a conversation with them as you would with the most “socially enabled” people (perhaps better, since most good programmers I know like their conversation to revolve around actually useful topics, rather than just inane banter).

Don’t ever hire a dumb person thinking they’re a good developer. They’re not. If you can’t have a great conversation with them in a relaxed social context, they’re very likely not a good programmer. On the other hand, anyone who’s clearly very smart at the very least has a strong potential to be a good or great programmer.

#4 : Hidden experience

This is correlated with the “Passion” point, but it is such a strong indicator that I’d like to emphasise it with its own point.

I started programming when I was about 9, on a Commodore 64. I then migrated onto the PC, did some Pascal. When I was 14 I wrote a raycasting engine in C and Assembler, spent a large amount of time playing with cool graphic effects that you could get your computer to do by messing directly with the video card. This was what I call my “coccoon stage”. When I entered that stage, I was a mediocre programmer, and lacked the confidence to do anything really complicated. When I finished it, I had gained that confidence. I knew that I could code pretty much anything so long as I put my mind to it.

Has that ever appeared on my CV? Nope.

I strongly believe that most good programmers will have a hidden iceberg or two like this that doesn’t appear on their CV or profile. Something they think isn’t really relevant, because it’s not “proper experience”, but which actually represents an awesome accomplishment. A good question to ask a potential “good programmer” in an interview would be “can you tell me about a personal project - even or especially one that’s completely irrelevant - that you did in your spare time, and that’s not on your CV?” If they can’t (unless their CV is 20 pages long), they’re probably not a good programmer. Even a programmer with an exhaustive CV will have some significant projects that are missing from there.

#5 : Variety of technologies

This one’s pretty simple. Because of the love of learning and toying with new technologies that comes with the package of being a “good programmer”, it’s inevitable that any “good programmer” over the age of 22 will be fluent in a dozen different technologies. They can’t help it. Learning a new technology is one of the most fun things a programmer with any passion can do. So they’ll do it all the time, and accumulate a portfolio of things they’ve “played around with”. They may not be experts at all of them, but all decent programmers will be fluent in a large inventory of unrelated technologies.

That “unrelated” bit is the subtle twist. Every half-decent java programmer will be able to list a set of technologies like “Java, J2EE, Ant, XML, SQL, Hibernate, Spring, Struts, EJB, Shell scripting”, etc.. But those are all part of the same technology stack, all directly related to each other. This is possibly hard to recognise for non-programmers, but it is possible to tell whether their technology stack is varied by talking to them about it, and asking them how the different technologies they know relate to each other. Over-specialisation in a single technology stack is an indicator of a not-so-good programmer.

Finally, if some of those technologies are at the bleeding edge, that’s a good positive indicator. For instance, today (November 2007), knowledge of Merb, Flex, RSpec, HAML, UJS, and many others… Please note that these are fairly closely related technologies, so in a couple of years, someone who knows all these will be equivalent to someone familiar with the Java stack listed in the previous paragraph.

Update: As a clarification to this point, there’s in fact two indicators here: variety and bleeding edge. Those are separate indicators. A good variety of technologies across a period of time is a positive indicator, whether or not the technologies are bleeding edge. And bleeding edge technologies are a positive indicator, whether or not there’s a variety of them.

#6 : Formal qualifications

This is more a of non-indicator than a counter-indicator. The key point to outline here is that formal qualifications don’t mean squat when you’re trying to recognise a good programmer. Many good programmers will have a degree in Computer Science. Many won’t. Certifications, like MCSE or SCJP or the like, don’t mean anything either. These are designed to be accessible and desirable to all. The only thing they indicate is a certain level of knowledge of a technology. They’re safeguards that allow technology recruitment people in large corporations to know “ok, this guy knows java, he’s got a certification to prove it” without having to interview them.

If you’re hiring for a small business, or you need really smart developers for a crack team that will implement agile development in your enterprise, you should disregard most formal qualifications as noise. They really don’t tell you very much about whether the programmer is good. Similarly, disregard age. Some programmers are awesome at 18. Others are awesome at 40. You can’t base your decisions about programmer quality on age (though you might decide to hire people around a certain age to have a better fit in the company; please do note that age discrimination is illegal in most countries!).

As a final note to this, in my experience most average or poor programmers start programming at university, for their Computer Science course. Most good programmers started programming long before, and the degree was just a natural continuation of their hobby. If your potential programmer didn’t do any programming before university, and all his experience starts when she got her first job, she’s probably not a good programmer.

Disclaimer

None of the indicators above or below are sure-fire indicators. You will find great programmers who break some of those moulds. However, my view is, you’ll rarely find a great programmer that breaks all of them. Similarly, you may find poor programmers that meet (or appear to meet) some of these criteria. But I do strongly believe that the more of these criteria a programmer meets, the more likely they are to be one of those elusive “good programmers” that, as a business guy, you need to partner with.

The criteria in bullets

So, in summary, here are some indicators and counter-indicators that should help you recognise a good programmer.

Positive indicators:

• Passionate about technology
• Programs as a hobby
• Will talk your ear off on a technical subject if encouraged
• Significant (and often numerous) personal side-projects over the years
• Learns new technologies on his/her own
• Opinionated about which technologies are better for various usages
• Very uncomfortable about the idea of working with a technology he doesn’t believe to be “right”
• Clearly smart, can have great conversations on a variety of topics
• Started programming long before university/work
• Has some hidden “icebergs”, large personal projects under the CV radar
• Knowledge of a large variety of unrelated technologies (may not be on CV)

Negative indicators:

• Programming is a day job
• Don’t really want to “talk shop”, even when encouraged to
• Learns new technologies in company-sponsored courses
• Happy to work with whatever technology you’ve picked, “all technologies are good”
• Doesn’t seem too smart
• Started programming at university
• All programming experience is on the CV
• Focused mainly on one or two technology stacks (e.g. everything to do with developing a java application), with no experience outside of it

I hope these help. Let me know below if you have any comments, or anything to add to them!

Thanks for reading.

Why have an estimation technique?

After four years in a large consultancy, it’s pretty obvious to me that you should generate proper estimates before you start on a piece of work. But, granted, that’s not obvious to all. Before I learnt how to do it, I too thought of “estimation” as some arcane art that you learnt over the years and that you couldn’t really explain to anyone. Not so. Say hello to the world of measurable, repeatable, somewhat accurate, and, most importantly, explainable estimation.

FPA is first of all repeatable. Once you’ve decided on how you measure function points, you can apply it over and over again, and constantly tweak your “coefficients”, to get better and better at doing estimates. This is because FPA estimates are easy to track against actual work, and adjust as you get more and more “data points”.

FPA is also fairly accurate. The main reason for this, in my opinion, is that FPA requires you to break down your work into small chunks and think about what goes into each chunk. That may still be wildly inaccurate - you may well have forgotten a huge component somewhere in there - but ultimately it’s way more accurate than a gut-level estimate. Funnily enough, I’ve found my gut-level estimates and my FPA estimate tend to agree, without me tweaking any numbers, even though they approach estimation from completely different angles (FPA = bottom-up, gut = top-down). Go figure.

Finally, FPA is explainable. It can be explained in excruciating detail to your clients, your manager, your employees, etc. It can be taught. It can be “institutionalised”. It’s a great people tool. With FPA, you don’t ever have to sit in a meeting, be told “well, I don’t think it should take this long”, and be unable to reply back with something really convincing.

What is FPA, then?

Function Point Analysis, despite the grand title, is a really simple technique. The core concept is easily represented in 4 simple steps:

1. Break down the piece of work in as many small chunks as you can (by screen, or by screen element.. whatever suits you - these are the rows of your estimates table
2. Pick some function points that make sense for your domain (more on this later) - these are the columns of your estimates table
3. Assign points from 0 to about 5 at most for each functional chunk - these are the values at the intersections between rows and columns
4. Add up the points for each functional part, maybe throwing in some sort of adjustment coefficient

What you get out of this is a very decent relative weighting of your various tasks. You can then pick a few really easy ones and use those to “calibrate” your coefficients, and thus automatically get more or less valid estimates for everything else.

Let’s go through a concrete, but very simple, example. I was asked to build a facebook application recently. Here are some of the chunks I broke it down into:

So far, nothing extraordinary. The next step is to pick some function points. You’ll probably only do this once for each “type” of project. For instance, facebook apps are all going to have more or less the same function points. In fact, most web apps will have more or less the same function points. My choice in this case is:

• Inputs - the number of ways the user can do stuff on the page/element
• Outputs - the number of dynamic things the page/element displays
• Exception conditions - the number of ways that the inputs can be rejected by the business logic
• UI Factors / AJAX - a general weighting factor to cater for the UI complexity and dynamic elements
• External integration factor - the number of external (ie not yours) systems the element has to integrate with

Another one I’ve found of some use is an “infrastructure” factor. That’s a general weighting factor to cater for the fact that some elements will require new infrastructure to be built and others won’t.

In assigning those points, you can use your gut to help you decide how many points to allocate. In my experience, you’re more likely to err on the side of putting too few points than too many, so it’s probably better if you err on the side of too many at first. You can quite easily go through and re-evaluate some outrageously large bits if you feel that they’re wrong. In all cases, ensure you are able to explain why you allocated a certain number of points to a certain element.

So, let’s chuck those in and assign some points:

The final step, adding those points up with some sort of coefficient, is the hardest on your first estimate, because you’ll really just have to wildly guess at the coefficients. But the beauty of having these estimates written down is you’ll be able to measure them up against actual work, and then you’ll have the full power of Excel’s Goal Seek functionality to allow you to fine-tune them to minimise the discrepancy. Here is my stab at it for this particular project:

As I hope you can see, this is actually a really straightforward technique to use. As soon as you start using it, you’ll gain much more confidence in your own estimates (if you were previously basically just guessing), and your estimates will start getting significantly better as you go through more and more projects and gather better data to help pin down those coefficients.

Final notes

Now, I can hear the hardcore technical project managers in the back start to boo me because I’m not following the letter of what FPA is supposed to be about. One of the key points hammered in in most of those over-complicated articles about FPA is that you should measure the function points very accurately, and always use the exact same function points for all the projects, etc. FPA is such a neat concept that all sorts of very smart people have gone and abstracted it in all sorts of ways to make it correct for “all projects”, but somehow the result of that abstraction, in my experience, has been that FPA has become hard to use, complicated, and has pushed accurate estimates back into the realms of arcane stuff that only a few people know about.

It doesn’t have to be so. As I hope I’ve shown in this article, FPA can be really simple so long as you use it as what it is meant to be - a much more accurate guess - rather than what it’s not - a foolproof technique to arrive at rock-solid estimates (my opinion about that: it doesn’t exist).

Feel free to post below and let me know what other estimation techniques you may have used, or even if you used FPA and found it useful. I’d love to hear about it.

Reducing code clarity

The most damaging side-effect of premature optimisation is a reduction of the clarity of the code. In my opinion, there are two main kinds of code performance optimisations available: high-level optimisations and low-level optimisations.

High-level optimisations generally revolve around rethinking the purpose of your code and whether it actually needs to do “the slow thing” at all. A good example would be turning 5 separate ORM queries that fetch a different object each time, into a single sql query that does all 5 joins in one go and so returns the end result in a single query. Adding a lookup table to an intensive loop is also something that I’d consider to be a high-level optimisation trick (you’re removing the need to perform a calculation over and over during the loop).

Low-level optimisations are those that don’t change the logic of the code, but only the implementation of it. For instance, rewriting a method in C or assembler to speed it up, unbundling a loop, or adding an index to a database table. Picking a language for development concerns is also a low-level optimisation. For instance, anyone who decides to write their code in C instead of Java “because it’s faster” is making that sort of early optimisation.

Both kinds (high- and low-level) tend to result in greater obfuscation of the code. That’s not always the case of course (adding an index obfuscates nothing, for instance), but it’s generally true. All of the examples above other than the index are optimisation tricks that make the code more complicated, and even more fragile.

This obfuscation becomes a problem when it’s out of control. If you’ve decided to optimise your code, you carefully identify the bottlenecks, and you build optimisations for just those bottlenecks, clearly identifying them from the “rest of the code” so that any future developer knows that they are basically hacks put in place for a good reason, there’s no problem. If, however, you have a collection of programmers making all sorts of optimisations left right and centre, as part of their coding style, you’ll greatly increase the rate at which your code base turns into a giant ball of spaghetti.

And the worst part is, of course, all those optimisations will amount to nothing.

Most optimisations don’t work

The worst thing about premature optimisations is that they simply don’t work. Here’s why. There is only one process that works for optimising an application. It goes like this:

1. Identify that there is a performance issue
2. Locate the cause of the performance issue and confirm that location through profiling and/or benchmarks
3. Apply patch to the cause of the performance issue
4. Measure that the performance fix produces improvements in the benchmarks

If you don’t follow these simple four steps, your optimisations are worthless. This is because performance is not a pervasive quality that lives in your code. You don’t “write high-performance code” in general and therefore get a high performance application. Most people see applications as a complex arrangement of gears which will turn faster if you put enough oil on it to lubricate the spinning. That’s an incorrect image.

From a performance perspective, an application is best represented as a series of bottlenecks. Improving performance is merely the process of removing the worst bottlenecks, in order of severity. If the code base is clean and well architected, this is usually fairly straightforward. There is a lot of literature out there about how to fix specific performance bottlenecks. Some of the bottlenecks may be tricky, but I’ve yet to find one where you couldn’t apply a fairly straightforward high- or low-level optimisation and remove the bottleneck.

Can you remove these bottlenecks early? Most of the time, no! Sure, there are a few instances where you might spot a very obvious “bad performer” and be able to tell, from experience, that this will be a problem later. But even in those cases, you should hesitate to optimise before you’ve put up some benchmarks. Even if you think you recognise a potential bottleneck, there is no guarantee that this is an actual bottleneck for the application. Since the optimisation is not free (it takes time, and it reduces readability of the code), you should never optimise before you identify something worth optimising (by finding a performance problem).

Caveat: architecting for performance

There are some situations where concerning yourself with performance early is valid. Generally, that time comes before you write a single line of code, when you’re still exploring possible languages and architectures for the application.

This is most often the case in two situations: low-level frameworks, or excessively slow dependencies.

If you’re writing a piece of low-level framework (like a 3D engine or a search engine - almost anything with the word “engine” in it in fact), you’ll have to make some high-level architectural decision to ensure that the required performance is possible. These should not be attempted by developers who do not already have a very good feel for what the performance bottlenecks are likely to be in this specific problem domain, however. If you’re thinking of writing a regular expression engine and you have no idea of what the bottlenecks are in the domain of string processing, give up and use/study someone else’s library until you do.

If you’re dealing with excessively slow dependencies (e.g. facebook queries, which can take half a second to return data), you must give some thought to how you will isolate yourself from these dependencies from an architectural point of view. For instance, within the realm of facebook apps, you need to ensure that your application never needs to make more than one or two queries to facebook to generate a page view.

In conclusion

If you’re building a web application of some sort, like 90% of all developers out there, don’t worry about performance until you must.

If you do hit a performance issue, measure it, fix it, and then measure the improvement.

If you’re building a low-level engine, consider performance in designing your architecture.

If you’re dealing with a very slow dependency, think about isolating your application from it through a well designed architecture.

Otherwise…

Stop Worrying About Performance!

DRY - Don’t Repeat Yourself

A key philosophy of Rails is to not repeat your code. Why? Repeated code (aka “wet” code) introduces the potential for bugs, takes longer to write, and is harder to read. If the code needs to change, and it’s repeated in multiple places, then you run the risk of forgetting to update one of the places where it’s been repeated. The result is hard-to-find bugs that take a long time to debug. Other key consequences of repeated code are: it takes longer to write (more things to type), it is more difficult to read (repetition clutters the code), it is more difficult to modify (you never know where to make the change to catch “all” the instances of the thing you want to change).

Keep the code simple and readable

Simple code is easier to read and understand. Therefore, it is easier for multiple people to work with. It is also easier for the original developer to go back to six months later. Finally, it is easier to discover bugs in simple code. Ruby and Rails both provide enormous potential for keeping code clean and simple, even while doing a lot of clever things with it. This is linked to the DRY principle, but subtly different.

MVC with thick models, thin controllers, thin views

Rails provides a full MVC stack, but that doesn’t mean that just writing in Rails will make your application a good MVC application. There are three places to put code in Rails: The model, the view, the controller. The view tells the server how to display the page - nothing more. The view should only ever contain simple display logic. The controller tells the server what functions to call in response to requests. It should only ever contain code that directs the execution towards the correct models and their appropriate method. The model tells the server what the business logic of the application is. Any validations, business logic units, etc, should go into this layer.

Test everything that you write (and only what you write)

Testing is essential for a variety of reasons. First, it provides a safety net to ensure that you don’t break one part of the code when you change another. Secondly, writing tests before writing code forces the developer to think about what they are developing before they start writing code. This always results in better designed, cleaner, more quickly written code. Ideally, the tests should consist of both BDD-type specifications that map directly to the code, and integration tests that check that the application is behaving correctly on the whole.

Tests are of key importance. Any code without an automated test suite is by definition unreliable and resistant to change, since you can have no confidence that changing one thing will not break three others. Testing enables you to be confident that you can make changes without making the code buggy.

Use Rails standard tools/code/plugins wherever possible

One of the advantages of Ruby on Rails is its huge community of developers. They produce many useful plugins which help accelerate development. Moreover, the core Rails distribution includes a large number of utilities (such as validators). These should be used whenever possible, in preference to custom code. Any custom code doing more than pure business logic or simple controller and view logic should be clearly justified.

Continuous unit testing and continuous integration

Tests and specifications should be executed as often as possible. Since the computer does not care how many times it runs the tests, there is no reason to settle for anything less than continuous testing. Tests should be running continuously in the background and notify the developer(s) immediately when they fail. The advantage of this is that developers know immediately when they write a piece of code that causes something else to break. This saves huge amounts of time in debugging.

What’s in an app?

That’s a key question when determining what to test and how. We’re looking at rails apps, here, so the basic model is MVC. So is this what’s in an app?

This is the basic model of a Rails app. However, from the perspective of deciding what to specify and test, it’s incomplete. I’m not talking about helper classes which belong in one of those three boxes. I’m not even talking about things like daemons, or mailers, or those kinds of components, because actually they still fit in the MVC model, just one without end-user interaction. What’s missing from this diagram, I believe, is the dependencies.

One of the problems which I (and many people, looking at the rspec-users mailing list and the #rspec channel on freenode) have hit many times and which is very irritating is what to do about those validators and relationships. How do I specify my polymorphic has_many :through, eh?

First, the big stuff

The first point I’d like to make in this article is that with the Story engine, we now have the tools to specify almost the entire system. The mapping is fairly simple:

First, you write Stories (in an outcome-focused way, with the “Given/When/Then” approach), which are essentially a much nicer way to express integration tests in a business-focused way.

Next, you write specification to define how your code will make those stories pass. You write a behaviour-focused specification each for the model, the view, and the controller, because these are the key architectural separations. Please note that this doesn’t imply you should write a specification for each method - specifications should be linked to behaviours, not low-level programming constructs. However, they are inevitably influenced by the architecture of the system.

Finally, once the specifications pass, the stories should pass too. If later, you refactor or change your code, you have your integration tests holding you up and making sure your system is still working as a whole.

You have two scenarios for a change: an internal change (e.g. refactoring) or an external change (change in the user requirements). The first is driven by a change in the code, the second is driven by a change in the Story.

If you change your code, the following things sequence of events should happen:

1. First, the spec for the item you changed will break. Start by fixing that spec to be correct.
2. Simultaneously, if you changed the behaviour, some of your Stories will break. This is not because they are wrong, though, only because your specs are mismatched - ie you’ve updated your spec for your item, but not for its dependencies. Don’t change your Stories at this point.
3. You proceed to update the specs for everything which was dependent on the item you changed. They break. You update the code. They pass. You’ll know that this process is complete because your Stories will pass again.

If you change the Stories, the process is subtly different:

1. If you change the Story, it will start by breaking.
2. You then change some specs to make the Story pass.
3. This may cause other Stories to break. At this point, you should not change those Stories (unless they are directly affected), but instead, fix the specs in the method outlined earlier, when looking at an internal change.

If your Stories are comprehensive enough, these processes will stop you from introducing bugs, while at the same time allowing you to spec the “right way” (at least according to me): by specifying behaviours rather than outcomes.

Issues with speccing validators and relationships

There are still some issues to do with validators and relationships. How should they be specced? The arguments tend to go like this:

Someone writes code like:

##################
describe User, "with neither facebook uid nor phone number set" do
  include UserSpecHelper

  before(:each) do
    @user = User.new
  end

  it "should not be valid" do
    @user.should_not be_valid
  end
end

##################
describe User, "with facebook uid" do
  include UserSpecHelper

  before(:each) do
    @user = User.new(valid_facebook_attributes)
  end

  it "should be valid" do
    @user.save!
    @user.should be_valid
  end

  it "should reject duplicate facebook uid" do
    @user.save
    @user_2 = User.new(valid_facebook_attributes)
    @user_2.should_not be_valid
  end
end

This prompts someone else to ask:

1 - What are you supposed to be testing?

The answer to this is usually “your own code”. Therefore, a test (I refuse to call it a specification) like the above makes no sense. Yet is the most common way that people specify a validator.

What am I testing here? The validator code. That’s really pointless, because Rails already tests that. And it results in a lot of duplication (though technically, it can be argued to be “not duplication” because you’re testing that the business rule of requiring a particular field is there, but I think that’s just rationalising a bad practice). Ultimately, though, this is uncomfortable because you’re testing an outcome, not specifying a behaviour. You may use rspec to do so, but it’s still an outcome. And that’s not what behaviour driven design is about. Ultimately, this is testing, not specification.

2 - But how do a specify one of those weird validator things?

I think a lot of the issue is also caused by the simple language constraint that specifying validators and relationships is just plain fugly at the moment. Here’s the code provided by David Chelimsky (chief maintainer of rspec):

it "should validate_presence_of digits" do
  PhoneNumber.expects(:validates_presence_of).with(:digits)
  load "#{RAILS_ROOT}/app/models/phone_number.rb"
end

Now I love rspec, and very much respect David - I’ve had a few chats with him on freenode#rspec and he’s a very smart guy - but this code is just nasty! That nastiness is required to get around the fact that the validator is loaded when the class loads, so can’t be specified as something that will happen later.

As per the Sapir-Whorf hypothesis that BDDers know and love, language drives behaviour, and nasty language like this doesn’t encourage people to go this way, even though, as I’ll explain a bit further down, I think this is actually the correct way to go, in spirit if not in form.

3 - But if I just specify that “thing”, how do I know that it actually does what it’s supposed to?

A couple of weeks ago I would have replied: This is not a valid concern. It just shows that you really need to write that spec. You need to find out how your code is going to work before you write it. Surely that should be obvious?

But then, like all things, this is not quite so black and white. There are validation and relationship constructs which are complex enough that you want to not only specify them, you also want to unit test that they work, and that they continue working even when things change, for instance when you deploy Rails 2.0. And that’s the key to solving this puzzle. The “specification” in point 1 looks wrong because it’s not a specification, it’s a unit test. It’s not an integration test, so it doesn’t fit in the Story Writer. But it’s also not a specification, so it doesn’t fit in your User spec.

So, the solution?

I put forward that this item belongs in a third category, that we thought eliminated by the arrival of BDD, but which actually should still exist: unit tests. Specifically, unit tests relating to the behaviour of third party code, or of your interpretation of that code. So these are not system integration tests, in the sense that they are not testing your system as a whole. They are more like unit integration tests, there to codify your understanding of the way the rest of the world behaves.

As such, they do not fit in the Story Runner. They have nothing to do with user requirements. What we need is a new directory in the rspec folder, dedicated to what I would like to call Assumption Tests. They are tests of your assumptions about your dependencies. As such, they are both redundant and essential. They are redundant in an ideal world where you know exactly how the external code you use works and you know about any changes to it. They are essential in the ugly world where you have to be just that little bit paranoid about other people’s code, even if it’s specified and tested up the wazoo, because sometimes things change in a way that breaks your assumptions.

And, importantly, these are not specifications. They are unashamedly outcome-driven. This is especially acceptable when you consider that they are not specifying your own code, but testing other people’s code to make sure it fits your assumptions.

Without further ado, here’s an example of one such assumption test:

class ActiveRecordTester < ActiveRecord::Base

  # This makes this model tableless
  def self.columns() @columns ||= []; end
  def self.column(name, sql_type = nil, default = nil, null = true)
    columns << ActiveRecord::ConnectionAdapters::Column.new(name.to_s, default, sql_type.to_s, null)
  end

  column :foo, :string

  validates_presence_of :foo
end

##################
describe "Presence Validator" do

  it "should not be valid without foo" do
    @test = ActiveRecordTester.new
    @test.should_not be_valid
  end

  it "should be valid with foo" do
    @test = ActiveRecordTester.new(:foo => "1")
    @test.should be_valid
  end

end

Once you’ve got this in place, you should never need to test the outcome of validatespresenceof again. So the code in example 1 is now obsolete, and your specifications can remain entirely behaviour-driven, rather than including bits and pieces of outcome testing. All you need to specify is that, say, your User model calls “validatespresenceof” for facebook_uid in the way that’s codified and auto-tested in your assumptions.

So now you have:

• specs for controllers
• specs for models
• specs for views
• stories for the whole system
• assumption tests for things which you didn’t write but want to test anyway

Sorry for the long article, but there were a fair number of ideas to cover. I’d love to hear comments about this, as it’s all still a bit fresh to me. You can put those comments below, or even comment on your own blog (I have trackback enabled). Thanks for reading!

Essentially, I decided to follow the advice given online. Don’t test the methods, test the behaviours you expect. Let’s take a simple example and see what this means in practice.

For our facebook application, I have a facebook controller which handles all the facebook posts (very RESTless, but that’s facebook for you!). There are two things that I want to see happen with every authenticated facebook action, and I’ve defined a filter for each of them:

class FacebookController < ApplicationController
  before_filter :require_facebook_install
  before_filter :set_user

  def index
    # ...
  end

  def other_action
    # ...
  end

private
  def set_user
    if @user = User.find_by_facebook_uid(fbparams["user"], :include => [:interactions])
      if fbparams["friends"]
        @user.cached_friend_uids = fbparams["friends"]
        @user.save
      end
      if fbparams["session_key"]
        @user