Inching Forward. Ish.

Bill and I met Tuesday to work on the iPad publication thing. We did a tiny bit more “design”, deciding that whatever the structure is in the Dropbox folder where articles will be placed, we’ll replicate to the articles folder in my site’s markdown source. This will “always” be two levels of folders, and will “never” go deeper than two.

Our little app will copy that Dropbox structure into the source site. Jekyll will build the site, and we’ll FTP the resulting files from Jekyll’s _site folder up to my web site. (This means our app will have to remember the folder structure so as to know what to push via FTP.)

Having more or less nailed down this bit of design, we needed to improve our tests to handle the multiple levels of folders. Remember that most of our testing hs the implementation right in the test. We’re trying to learn how best to use the Dir and FTP classes, and to have the test serve as documentation for when we forget. Which will be by tomorrow. In fact I’ve already forgotten a lot of it.

Last time we used long and explicit folder paths. That was ugly, ignorant, inconvenient, and hard to change. It did, however, work. So much of what we did this time was to use the default folders, and folder setting, to our advantage. Let’s look at the tests. I’ll intersperse some commentary.

The Code

require "minitest/autorun"
require "net/ftp"
require "./passwords"

class Test_FTP  < Minitest::Test

  def setup
    FileUtils.rm_rf("./_target")
    FileUtils.mkdir("./_target")
  end

Our earlier setup was very ad hoc. This one simply removes the test’s target folder recursively, and then creates it empty.

  def test_target_is_empty
    target = Dir.glob('_target/*') 
    assert_equal(0,target.length)
  end

This just verifies that the setup worked, checking to be sure there’s nothing in target.

  def test_connect_finds_no_files
    ftp = ftp_connected_to_target
    list = ftp.list('*')
    ftp.close
    assert_equal(0,list.length)
  end

This tests that, upon startup, an FTP looking at our target directory finds no files. This begs the following definition:

  def ftp_connected_to_target
    ftp = Net::FTP.new('localhost')
    ftp.login(Passwords::USER,Passwords::PASSWORD)
    ftp.chdir('programming/test_ftp/_target')
    ftp
  end

Nothing to see here. We init an FTP object and point it to our ftp target folder. The test above, which doesn’t put anything there, just checks to be sure there’s nothing there. This test is arguably redundant with the preceding one, but it tests, weakly, whether we have the FTP object pointing to the right place.

So far, all of this is just scaffolding to be sure that we are set up right and have our heads more or less screwed on. Now we actually move some files:

  def test_upload_files
    ftp = ftp_connected_to_target
    oldDir = Dir.pwd
    Dir.chdir("/Users/ron/Dropbox/_source")
    source = Dir.glob('**/*').sort
    source.each do |file_name|
      if File::directory? file_name
        ftp.mkdir file_name
      else
        ftp.puttextfile(file_name,file_name)
      end
    end
    ftp.close
    Dir.chdir(oldDir)
    Dir.chdir("_target")
    target = Dir.glob('**/*').sort
    assert_equal(source, target)
    Dir.chdir(oldDir)
  end

OK, here there’s some meat at last. We connect our FTP, save the current folder that Dir is looking at, then change directory to our source folder. We note that this name is hard-coded in and that it’ll surely want to be made more flexible somehow. That’s for later.

Then we get a list of all the folders and files below our source folder. We sort because of what follows: we want all the folders listed before the files and /mumble/foo sorts ahead of /mumble/foo/index.md.

Now we loop over that list and if the list item (OK, file_name isn’t a great name here) anyway if the list item is a folder we create the folder in our target, using FTP’s mkdir. N.B. This isn’t good enough for the real thing. The folder might be there already, from a previous run. If so, we want to move our files into it but we can’t create it because it’s already there. Make a note of that, Bill.

If the item isn’t a folder, it’s a file, and we put it across. We’re only putting as text. We have in mind to test an image file to see what happens, and we’re really hoping we don’t have to decide which is which.

Having put them all across, we just do a glob on the target folder and clean up. We also restore the Dir’s default folder, since we changed it. We think that would likely break subsequent tests if we didn’t do it. And we think it should be done in setup and teardown, not here.

  def test_make_subfolder
    ftp = ftp_connected_to_target
    ftp.mkdir("subfolder")
    ftp.mkdir("otherfolder")
    ftp.mkdir("otherfolder/subsubfolder")
    ftp.puttextfile("test-ftp.rb","otherfolder/subsubfolder/RUBY.rb")
    ftp.close
    exists = Dir.exist?("_target/subfolder")
    assert(exists,"folder not made")
    exists = Dir.exist?("_target/otherfolder/subsubfolder")
    assert(exists,"subfolder not made")
  end
end

And here’s our last test. It is documenting our understanding of FTP’s folder making ability. Note that before we make otherfolder/subsubfolder, we make otherfolder. If we don’t do this, the test fails. This is consistent with how Unix and such work: you can’t make a folder chain all in one go. Someone, somewhere, decided this. I believe they were not as right as they might have been, but anyway this is how it works.

Discussion

One of my missions over the past few months, and going forward, is to document how many “Erors” I make in programming. My theory is that I don’t make many more than anyone else, and my hope is that when folks read this, they’ll feel more OK recognizing that we make lots of mistakes and that it’s good to be aware of them and open about them. Another possibility is that folks will think “Well, at least I’m not that stupid” and feel better that way. It’s all good.

Now anyone who uses Ruby’s Dir and FTP a lot should have all this at their fingertips. If I were a woman, or not quite as scary a man, some of you would be mansplaining me to death and telling me how ignorant I am.¹ Be that as it may, we have not used them a lot and we fully expect to forget before next time, so we’re happy to have our learning documented in these tests.

You can be sure that we made many mistakes getting these to work, between not understanding how these objects are meant to work, trying things taht we thought “should” work, trying things that we thought “shouldn’t” work to be sure they didn’t, and generally typing garbage into the strings.

There are also some Erors™ in what’s left, or at least infelicities. The big test needs refactoring, since it has several ideas in it, as we saw when I tried to explain it. Those ideas should be expressed in the code.

There need to be more tests, and tests with better names, such as test_Dir_defaults_to_root or whatever Dir defaults to. Generally speaking, we need to look at each of those tests and think about what we were trying to find out, and what we did find out, then name the test accordingly.

Looking Ahead

We think we know enough now to try to write an actual acceptance test that copies files down from Dropbox into a Jekyll source folder, runs Jekyll, then copies the resulting files to FTP. I was arguing to fake the Jekyll step on the grounds that it would take too long to set up Jekyll, but Tozier convinced me (by doing it) that the basic setup won’t be too arduous. I still think we’ll find it a bit tedious setting up the test but with luck we can do it once and for all, by copying things from my real site. And it’s certainly the thing to do if it isn’t too expensive to be worth the trouble.

This test will be quite important, since I am really quite reluctant to set an automated bandsaw loose on my web site. I’m not sure why I feel that way but I think it has to do with a half-century of Erors™.

Stay tuned!