Between the new book review page and a few other things, I just spent a few days in a painful, test-free update of my web site scripts. Here, a report on how much it hurt. Lessons learned? We'll have to see what I do. Like lots of folks I know, I haven't figured out how to do what I know I should.

But the Results Are Good

I’m inordinately if not unduly proud of the new book review page, but there is a dark side. Let me tell you first why it’s cool.

XSLT, you probably know, is an XML-based language for transforming XML documents into other forms. It can produce almost anything: text, HTML, PDF, whatever you want. I use it to translate my articles, which are written in a simple XML form, into XHTML for the web site. XSL lets me change the format of things without having to edit the original files.

The process is mediated by an Ant script with about 20 targets. The core target is the one that translates the articles, but there are many others, such as the ones that produce the indexes, done with a mix of Ruby code and more XSL. The Ruby indexing code goes through all the articles, finds the ones of interest, based on an internal tag that says whether it’s about documentation or whatever, and appends all the header information into one big file. This is done because XSLT processors only process one input file at a time, so if you want an index page, you need a single file with all the information.

The normal indexes are fairly simple: I produce the combined header file, then sort it with XSL in inverse date order, then run that through the formatter.

In all of this, the XSL is fairly straightforward, if XSL is ever really straightforward. A typical XSL “program” is rather non-procedural. It is made up of “templates” that match patterns in the input XML, and produce output according to the details inside those tags. Here’s a simple example:

<xsl:template match="newcode">
 <span class="codehighlight">

That template is called whenever XSL encounters a tag that looks like <newcode>. What the template does is emit the span line calling for class=”codehighlight”, then apply templates, then end the span tag. The apply-templates call tells the XSL processor to read whatever’s inside and process it however the rest of the program wishes. So if inside there was, say, a <bold> tag, whatever XSL handles that would handle it.

The result is that no matter how complex the stuff inside the <newcode> tag is, it will be correctly processed, and all the resulting tags will nest and un-nest properly.

The Book Reviews

My site’s book reviews have a special tag in their XML source: <book>. This tag includes title, author, class, asin, and a short blurb for the book. The class is any string: right now we have “agile”, “programming”, and so on, as you see on the site. The asin is used to connect the review to the picture of the book, and to link it to the appropriate page on Amazon. (Feel free to click through and buy your books that way. It won’t cost you any more, and I’ll get a few cents. I’m sure you think this stuff is worth that if you’re still reading.)

I wanted the book review page to display the books in that nice 3-up table, organized by class. I wanted books to be able to appear in more than one class. (See if you can figure out what book I had in mind when I decided that.) There were a number of challenging problems in this simple spec.

First of all, there was the three column table. XSL can express a pattern to be executed every time it sees a book tag. But the table processes three books at a time, and it’s very difficult – I never figured out a way – to retain enough state to tell XSL when and how to emit the table row (TR) tags. One reason this is hard is that you can’t write two different patterns, one to open a tag and one to close it: every block of XSL code must be a balanced chunk of tags.

Basically, what you want to do is to loop over a collection of book tags, incrementing by three. You’d think that would be easy. You’d be wrong.

There is no looping capability in XSLT, other than to loop, by ones, over a collection. It has <xsl:for-each> and nothing else. What’s a girl to do? Well, you Google around for things like “XSL loop table” until you find the trick. The trick, of course, is tail recursion.

You probably know what tail recursion is. It’s that creative way of making a loop by calling yourself. We “just” need to have a template that displays three cells, then calls itself pointing to the next cell. Arrgh. But it works, and it looks like this:

<xsl:template name="displayRow"> <!-- loops using recursion -->
    <xsl:param name="pos"/>
    <xsl:param name="list"/>
    <tr valign="top">
        <xsl:call-template name="displayCell">
            <xsl:with-param name="pos" select="$pos"/>
            <xsl:with-param name="list" select="$list"/>
        <xsl:call-template name="displayCell">
            <xsl:with-param name="pos" select="$pos+1"/>
            <xsl:with-param name="list" select="$list"/>
        <xsl:call-template name="displayCell">
            <xsl:with-param name="pos" select="$pos+2"/>
            <xsl:with-param name="list" select="$list"/>
    <!-- for some reason we have to use this temp.
         if we don't, we get an exception in the transform -->
    <xsl:variable name="BOOK" select="$list[$pos+3]"/>
    <xsl:if test="$BOOK">
        <xsl:call-template name="displayRow"> <!-- recursive call -->
            <xsl:with-param name="pos" select="$pos+3"/>
            <xsl:with-param name="list" select="$list"/>
Refactoring note: This would be more clear, I think, if I named the list parameter first, and then the position. And probably spelling out position would be best as well.

Nothing to it. We enter with an integer, named “pos”, and the list of books we want to display. Then we call displayCell three times, with the list, and with pos, pos+1, and pos+2. Then if the book at pos+3 exists, we call ourselves recursively with pos+3, and the list.

The trick is making this work. Outside somewhere, we’re calling in with a table tag and a table row tag, of course. That’s pretty reasonable. Inside, we start with something really simple, instead of that displayCell call: something like this, maybe:

<td>Number: <xsl:value-of select="$pos"/></td>

That’ll display a simple table row of integers, which is enough to tell us how we’re doing. After that, we just elaborate until we have what we want.

And That's Why It's Hackery

So the situation I found myself in was that I really didn’t know what the HTML should be to look right on the page, and I was building up the XSL bit by bit until the page looked right. Edit, build the site, look at the page. Repeat. Repeat repeat repeat.

When I build code up that way, it tends to get really long and weird and procedural. Maybe that has happened to you as well? The code shown above has the template “displayCell” broken out as a function. But when I start with the little TDs and such, I’m more likely just to start filling in, until the code gets grubby enough to notice. I actually re-invented the Extract Method refactoring while I was doing this, when it got too ugly to understand and I realized I could extract a template like “displayCell”.

But what about all those tests? What about Test-Driven Development?

Well, on the one hand, I was testing a lot, but it was all manually, run the site build and look at the result. But that’s not a lot like calling your shot in a TDD test and then making it happen. Why did I fall off the wagon?

Why Didn't I Test?

My site was being built in Ant, running XALAN, a Java-based XSL processor. I could, in principle, set up a couple of files and run XALAN on them. Maybe a little batch file or something. But comparing the results would be hard.

The site code also uses Ruby, which has an XSL processor too. I could write little Ruby tests, feeding in input to the Ruby processor, and then when it worked, maybe move the code back to my main site-building location. I’d have to learn how to do that, and probably have to download at least Ruby’s XSL processor. Even then, it wouldn’t be the same processor as the site uses, so I would worry that things would work in one and not in the other.

Microsoft .NET has an XSL processor. I could write NUnit tests to drive it. Again, I’d have to learn how to use it. I do have some examples that I wrote a while back, so it would be possible. But again, it’s a different processor.

XALAN is written in Java. I could fire up Eclipse, jUnit, XALAN, and write jUnit tests in a style more or less like I’m used to. But then I’d have to make Java and Eclipse work on my machine.

Any of these things would have been possible. But each one would take a bunch of hassle downloading and learning things that weren’t related to the work at hand, which was to get the book review page working. It would be a big chunk of tool work and study, for a very limited payoff.

This, of course, is what keeps people from testing their code: the first test is really hard to write. If I would write the first test, the second one would be easy. There would be odd things to figure out, like how to get the tested code included back into the main line, but those are all solvable. In fact, XSL has an “include” feature that would let me just include the tested code directly, without changing it at all.

The first test is the hardest, and the most important. XSL is a perfect example of a language where you can only debug it by running the program and seeing what it does. You can try to write modular code, try to code by intention, but when it comes down to it, unless you build a framework, tests are hard to do.

Not an excuse, mind you, but an explanation. I’m sure you understand: you’ve been there – you might even be there with some of your own code right now.

But the lessons get harder:

Grouping by Class

My original effort got all the books displayed in alphabetical order, but I wanted the display to be grouped by class, as it is now. I worked up to that in stages, still with the same approach of making a small change and looking at the results, sometimes by displaying trace information on the page, sometimes just by looking at the real page effects. I started by just typing in the then-current book classes, “agile”, “programming”, and so on, and making a new template that selected just those books. It looked roughly like this:

<xsl:template name="displayTableByClass">
    <xsl:param name="className"/>
        <xsl:attribute name="name">
            <xsl:value-of select="$className"/>
    <H3>Reviews classified as '<xsl:value-of select="$className"/>':</H3>
    <xsl:variable name="list" select='//book[class=$className]'/>
    <xsl:call-template name="displayTable">
        <xsl:with-param name="bookList" select="$list"/>

Note the creation of the variable name “list”. That takes the className and selects all the books having that class as one of their classes. It took ages of searching and trying to find that creation. The issue revolved around the “//”, which is the code for starting at the outside of the document to search for books. It should have been obvious that we needed that, but the path I took to get there obscured what was happening.

This is a place where a test focus would really have helped. The issue in hand was only selecting the right books, given a class name. It would have been easy, had I had a testing fixture set up, to write a tiny test and try various path syntax until I found the one that worked. As it was, every test took about 30 seconds to do, and each time there was a chance that the XSL wouldn’t even parse correctly. On the bright side, the display from running the whole blob gave pretty good diagnostics of what was wrong, and the test fixture might not have. Note that I say “might”, because I don’t know.

Getting the Classes

The next trick was to get the classes out of the book list itself. Each book has one or more classes, and we need a list with each class occurring once, and we want to process it in alphabetical order. (Really, I’d have liked to process them in alphabetical order except with “other” last in the list, but that seemed to be asking too much. Probably could do it with a lookup table or something … maybe later. The code to create and use the list looks like this:

<xsl:template name="displayAllClasses">
    <xsl:for-each select="//page/book/class[not(.=preceding::class)]">
        <xsl:sort select="."/>
        <xsl:for-each select=".">
            <xsl:call-template name="displayTableByClass">
                <xsl:with-param name="className" select="."/>

Note the first for-each, with the select: “//page/book/class[not(.=preceding::class)]. That selects classes, starting from the root of the document, such that the class does not occur in any of the preceding classes. Apparently. The notation is certainly cryptic enough for most purposes, but it’s one of those things that you learn, and after that you sort of know how to do it. Note also, in that patch of code, the use of the symbol “.”. Dot is analogous to its use in file directories or folders: it means “the current one”. So each of the dots means something different:

  1. In the first for-each, it means "whichever class you're considering selecting";
  2. In the sort, it means "whichever class you're looking at, from the group selected above";
  3. In the second for-each, it means "the now-sorted list".
  4. In the with-param, it means "whichever class we're currently processing in the loop".

At least, that’s my understanding of what the dot refers to at each of those points, based on lots of testing with little print statements to see what came out at each point. I think I’m roughly right, if not exactly right.

This is another place where some testing would be handy. We’ve probably all tried to figure out some new language or API by tweaking statements to see what they do. Probably six useful things went by while I was looking for the right way to write those statements. Had I been writing tests, many of those useful things could have been preserved. All I’d have to do would be to rename each test to have a name reflecting what actually came out, save it, and write a new one looking for what I wanted. The result would be a test showing how to find all the other stuff I found, such as “the books belonging to this class”, “the first book belonging to this class”, and so on.

It Only Gets Worse

Owing to another issue that I’ll spare you, I went looking for an Ant capability that my version didn’t have. It appeared that to do what I want, I needed to upgrade my Ant to the latest release. No problem. I backed up my existing Ant folder (not entirely a fool here), brought down the new stuff, and installed it. A bit of messing with environment variables, and it was time to run the site build script.

The first thing that happened was that it tried to build the entire site, not just changed articles. I finally figured out that the Ant “style” task had been changed to make running dependent on the XSL being newer than the eventual output files, not just the input files. That’s a righteous change, even though it meant it would build all hundred plus articles for me.

Except, of course, that some of the articles wouldn’t transform: the new XSL processor was incompatible with the old one. It didn’t allow multiple occurrences of the same template, which the previous one had, nor did it allow a missing template to be treated as an empty one, which it appears the previous one did as well. And there were a few oddities in the input files as well. I probably had to make a dozen changes, some of which were pretty tricky, but none took more than perhaps a half hour to figure out.

It was just a matter of running the build, seeing which file broke, and fixing the problem. Standard conversion stuff. Probably no real point to having tests here, it was a case of needing the real programs all in place.

After the pages all translated, a few other differences showed up in Ant. Some of my targets were using now-deprecated syntax. Some of those changes were fatal and I had to fix them … others remain to be fixed later.

Finally, we were down to the final chapter, the creation of the book index, which runs last, as an artifact of the way the targets are set. It failed, with a fatal null pointer exception in the XSL translator!

Arrgh! If I couldn’t find and fix that, I was doomed and this whole exercise was a waste of time.

Binary Search for the Bug

When you have an unknown problem causing a compiler or such to explode, without giving you a decent diagnostic of where it is, there’s this trick where you comment out half the program and see if the problem still occurs. If it does, you comment out half the remaining half, if not, you comment out the other hand and carry on like that.

In a highly-structured language, of course, you have to comment judiciously, so as not to ruin the structure of the program. So I proceeded that way. I was able fairly quickly to narrow it down to the code listed up above, with the comment about the exception. Here it is again, the old way, and the new way:

    <xsl:if test="$list[$pos+3]">
        <xsl:call-template name="displayRow">
            <xsl:with-param name="pos" select="$pos+3"/>
            <xsl:with-param name="list" select="$list"/>
    <xsl:variable name="BOOK" select="$list[$pos+3]"/>
    <xsl:if test="$BOOK">
        <xsl:call-template name="displayRow">
            <xsl:with-param name="pos" select="$pos+3"/>
            <xsl:with-param name="list" select="$list"/>

I was kind of aiming at the problem being here, based on a couple of probes, and because the tail recursion is so odd a technique. But if it didn’t work, I was well and truly doomed.

Fortunately, the first thing I tried, breaking out the test expression into its own variable, worked. The problem was solved, and the whole site builds correctly as far as I can tell.

And Then ...

I now have a perfect bug report for someone. A one line change that is the difference between a program that works, and a null pointer exception. It would be easy enough for me to trim the program way down, as well … if there were anyone who wanted this report, and I could find them. That turns out not to be the case.

Ant ships, it appears, with its own copy of Xalan, but I’m not sure of that. It took me about 12 hours to get signed up for the Ant mailing list and to get a message back from someone telling me how I might be able to find a newer version of Xalan, which I could of course do by signing up for the Xalan project, downloading some random jar file, putting it in some random location, and so on. Then, if it worked, I would know the bug was fixed and could stop. If it didn’t, I might want to do the same thing with Xerces, the XML parser that Xalan uses.

My guess is that would take me somewhere in the range of a complete working day just to find out whether I need a bug report at all, and then I’d still have to submit it. If I were a better man, I’d put in that effort.

As it stands, I have a better idea. I’ll post what I know in this article, and see if I can leave some seeds leading to it where the Ant/Xalan/Xerces community will find it. If anyone gets in touch with me, I’ll work with them to find the problem.

Meanwhile, we have lessons to learn.

What to Learn, What to Do?

In doing the Grotesque Hackery described here, it went as it usually does. Sometimes it was going well, sometimes I was on the verge of despair. I stayed up too late looking for that last bug. I came close to wrecking everything, and had to back up the code a time or two. All the stuff that always happens. The ability to write small tests would have helped a lot.

But I don’t have that ability. Should I build it? Would it pay off?

I don’t update my site generation software very frequently at all. Still, there are a few things on my list, and probably I’ll think of more when next Anna Lissa Cruz, who helps me with some of this, becomes available to work on the site again. Most of the changes are very visual in nature, of course, and they all yield to doing things incrementally.

One thing that I know will be a bit large is that I’d like to improve the site indexing. There are quite a few unindexed topics already, and the site could probably benefit from a better breakdown. That change will be interesting because the new style task in Ant can create more than one output file from one input file. In the previous version, to create two indexes, I had to make copies of the input files, because the output file name had to match the input file name. With that no longer necessary, I can simplify the flow of the Ant tasks. On the other hand, how do you test an Ant build? I sure don’t know.

It’s possible that I’ll decide to build an index of indexes page, one that shows all the different indexes on one page, a bit like the way all the book reviews go on one page. I’d probably keep the existing article precis paragraphs as they are, but would want to group the articles by topic, as the books are grouped by class. Each set of articles should have a kind of general introduction, I’d think, talking about the topic area overall, before listing the individual articles in that topic. That would imply some kind of lookup to find those paragraphs. And where will I keep them? In the XSL files? Elsewhere? What about a topic with no description?

I’ve been considering having the home page randomly suggest a link to one of a list of “recommended” articles, ones that have held up particularly well over time. Which reminds me, the master index should handle duplicate topics, so I can add the topic “recommended” to articles that already have another topic.

The simple index will require changes to the XSL files, though it will probably mostly be copy-paste, and the complex index will be even more complicated than the book review page. It, surely, should be tested.

What Might One Want to Test?

Let’s muse on what we’d test if we knew how.

I’d like to know that all the necessary things are built, in the right order. That could be expressed as assertions like these:

  • For all x.xml, x.htm is newer than x.xml.
  • bookindex.htm is newer than any of (topicacsindex.htm, ...) which are all newer than any other htm files.
  • The booklist for the book randomizer, and the book randomizer itself, are newer than all the articles.
  • Maybe: intermediate files are older than the files that depend on them.

However, the ant script rarely changes. It only changed this time because a new Ant version broke it. (Grr.) So these tests, while nice, don’t seem to be of much value.

I’d also like to know that the XML-to-HTML transformations “work”. That means, that they create HTML that will look the way the web site is supposed to look. Could I break that down into smaller notions?

  • The main site's three-column with header layout is bunch of XML to create a literal, in essence. It's not clear how you test a literal. Type it in twice?
  • It might be nice to test each XML element that I use in articles to be sure that it creates the right string, like the "newcode" tag mentioned above.

I don’t see much here; I don’t see the payoff. Problem is, the clients I talk to don’t see much payoff for the tests I recommend either, but when they break down and do them, they get great payoff, and make better decisions thereafter.

It might follow that I should drink my own koolaid and test this Ant/XSL stuff, just to learn what would happen if I test it.

It’s pretty easy to see from here that having some testing ability in place will pay off when I get around to doing this work. But is it YAGNI? I know I’m going to need it, but I don’t need it right now. Should I invest in building up this technical ability, against the day when I’ll need it?

That’s a tough call, and it touches on a lot of the things I teach in my approach to this Agile stuff: when do we invest code today that won’t be used until some future time? In general, I rail against putting in features that we don’t need yet. Is this one of those cases? I can tell this is going to need another article someday.

But right now, freshly healing from the bruises of the last couple of days, I know I’d have done a better job of this had I been a bit more skilled. Skill-building is always on the agenda: it’s a question of what to invest in. It would be good to get my site-building scripts under test, and of course it’s always better to preach from a position of high moral standing.

I’ll chat with Chet on this over the next little while, and listen for ideas from the lists and email, and see whether I can muster the gumption to build up the start of a testing setup. Stay tuned.