Stop Calling Your Tests “Stories”

Stop calling your tests “stories.” Really.

They’re not “stories”. They’re tests. They don’t tuck you into bed with a glass of warm milk and a nice fairytale to dream about; they assert that certain things should happen and complain when they don’t, in a purely mechanical and precise way. We’re programming computers, machines of pure logic here, not reading to children. Let’s dispense with the pointless cutesy fa├žade and focus precisely on the task at hand.

They don’t deliver what they promise, anyway. Frameworks like RSpec and Cucumber do not let you write tests in “plain English”. They let you write tests in a horrible chimeric programming language that adopts a superficial English-like veneer over top of what’s essentially a regular old unit testing framework.

It winds up giving you the benefits of neither English nor a well written testing API; your code simply becomes excessively verbose for the sake of a (highly fake and brittle) natural language veneer. You’re still highly restricted in what you can say, just like a regular old programming language. The set of Englishisms that you can use is so grossly restricted and precisely defined that you can’t think in anything remotely like English. You have to memorize a precise set of rigid, fixed English labels and use that and only that. You do get the verbosity of natural language, though, along with the restricted semantic domain of the API. You get the benefits of neither and the disadvantages of both.

Since you’re memorizing a highly restricted API anyway, skip the twee cruft and just use a short and precise API from the start, without the pseudo-natural language clutter.

8 Responses to Stop Calling Your Tests “Stories”

  1. Hi there,
    How about using the word check for what you describe in your post. Because that is exactly what they are doing, check that a predefined input gives a predefined output. When testing something, the evaluation of the result is a sapient decision if everything looks ok or not. But what you have in your code are checks.

    See what I write about it here:
    http://happytesting.wordpress.com/2011/03/29/good-programmers-check-great-ones-test-as-well/

    and the original post by Michael Bolton here:
    http://www.developsense.com/blog/2009/08/testing-vs-checking/

    But you are right, Stories are even more strange wording for it.

    @Siggeb

  2. I agree with @Siggeb that ‘test’ is misleading by modern standards. Any ‘testing’ you did (sapient process, questing something, evaluating results) happened as you were writing the code to check your program. As soon as you re-run that code, it becomes a check.

    Still, we’re kinda stuck with that legacy, so being specific and calling them ‘Unit Tests’ is OK in my book.

    I agree with you that ‘stories’ isn’t really meaningful when you’re describing unit tests.

    The rest of this post seems to be a thinly veiled jab at test frameworks employing ‘english-like’ syntax to improve readability and understanding.

    Presumably it’s your personal preference to use more terse frameworks and nothing to do with calling things ‘stories’.

    In RSpec parlance, they’re ‘examples’, not stories, which is actually a pretty descriptive, meaningful word for the way RSpec is ideally used; to express examples of how your code works.

    In Cucumber parlance, they’re ‘features’, not stories. From a User perspective, where Cucumber is most effective, ‘features’ are probably more accurate.

    • James,

      It wasn’t thinly veiled at all; it was a quite explicit jab at these attempts to use English-like syntax to purportedly improve readability and understanding. I elaborate on the reasons why in the post.

      Paul

  3. I’ve never heard this usage before.

    At my employer, a “story” is not a test but a minimal unit of business value. A story needs tests to verify that it is complete; but a story is not the same as a test at all.

    • That’s just it. You can indeed make a plausible case for calling a relatively high level product design element written in natural language a “story” (though I would argue against it personally, as a “story” implies that it is of questionable or at least unverified veracity.) But many “behavior driven” testing methodologies and libraries take this already questionable metaphor too far (e.g. http://dannorth.net/whats-in-a-story/ and similar).

      RSpec and Cucumber come to mind as the worst offenders, because they try to shoehorn the tests themselves into a grossly artificial pseudo-English “story” paradigm. That doesn’t make the tests any more readable or understandable (since they’re not actually real English); it just makes them needlessly verbose and brittle without adding anything at all in the way of functionality.

      Proponents even sometimes claim that they let you write “plain text stories” as their primary feature. Cucumber bills itself as “BDD that talks to domain experts first and code second.” No domain expert I’ve ever met talks in anything remotely like the constrained and stilted trivial subset of English that Cucumber provides.

      Programmers will be irritated by the childish pseudo-English code, and domain experts will be irritated by the artificial, massively constrained syntax and vocabulary (which is really just an unwieldy isomorphic transformation of a regular old programming language.) It is a wildly unsatisfactory attempt to compromise between code and natural language that only winds up failing at both.

      Granted, it would be great if we had a functional natural language parser which we could use to write our specifications in regular English or some other human language, but we don’t, so best not to pretend as though we did.

      Paul

  4. This complaint completely misses the point of frameworks like RSpec and Cucumber. It isn’t that you can write them in English, it is that you can read them in English.

    In one sense, these tests are stories – stories that the developer is telling the product manager / domain expert / user representative / whatever-you-call-him-or-her.

    At the same time, they are also a set of precise, mechanical assertions of the way the system behave. As such they are as hard to write as any other piece of code.

    Nobody ever found a way to code using “plain English”, and the RSpec/Cucumber people weren’t silly enough to try. However, making specification/tests/stories code readable enough to be useful in communicating with non-developers – that is a significant achievement.

    • But they aren’t readable English. Sure, they get into the general vicinity, but they fall far short. They’re stilted, constrained pseudo-English which is also an excessively verbose piece of code, gaining the drawbacks of both. No domain expert, etc. talks like that. They are forced to learn and comprehend this artificial toy subset of English, to force their thoughts into its minimalistic channels, when both reading and writing it.

      Non-developers have no need to be able to read the actual tests. That is the root of the issue; some people apparently think they should. Non-developers should, of course, be involved in the design and specification, but using “real English,” precise but natural human language statements of what the software should do. It is then up to the developers to translate that into code. The chimeric bridging mechanism between these domains proposed by RSpec/Cucumber only makes things harder for everyone, on both sides, while adding little or nothing. It’s a nice thought, but wait 20 or 30 years for natural language parsers to become useful first.

      Besides, the word “story” means “an anecdote where the entertainment value is more important than whether it is literally true.” That’s fine for writing a novel or making a movie, but it’s a horrible way to design software. Software is an inherently precise matter in every aspect. If I’m funding or working on or going to use a piece of software, I don’t want something in it that was built around entertainment rather than truth. Why would I build something if it’s not necessarily true that a user needs it? Software should be designed around precisely specified “use cases” or the like, not “stories.”

      I think part of the problem is that many people find it difficult to write precisely yet in non-formulaic natural language. Schools should teach philosophy more…

      Paul

  5. A bit late to the party here but the origin of the term is from the agile/UX world where requirements will often be described as “user stories”. I presume that the theory is that UX people could colaborate on testing by mapping their stories onto user tests and then understand those tests. How well it works…well with a large part of poor software design being poor requirements, anything that puts those requirements under a brighter spotlight is good in my book. And conditioning people to think of requirements as concepts that should be testable in their own right helps to crystalise peoples thought processes about the entire requirements proceedure. But yes, the language isnt quite there yet to “naturally” bridge that gap, but its a good first step rather than just putting up with the vague crap that often passes as a requirements specification (or collection of user stories of you prefer).