Very Nice Idea
Hello, loves!
Better yet, it will involve actual programming! Let’s improve the Making App.
My brother GeePaw Hill refers to the Making App and the Shipping App as aspects of a software product. The Shipping App is, of course, the bits you ship to the users. The Making App, in contrast, refers to the things that we build into the product that help us to build the product. I have a very nice idea for the Making side of things, and it promises to be both useful and fun to build.
We’ve been working off and on on one of our Making requirements, around the BuildStepper, which lets a developer step through the creation steps that build up the dungeon, so as to see the effect of each step. It’s aimed at one of the teams in our large dungeon product development, the individuals who create layouts and populate them with denizens and curiosa. Of course, there’s only me here. We are using this imaginary dungeon product as a basis for setting up and solving programming issues that arise in real software development, as done by humans.
The BuildStepper is simple and somewhat useful. It is given a list of functions that called in order, and the functions do useful things as part of building up the dungeon. A stepper list looks like this:
steps = [
bottom_left,
make_a_diamond_room,
make_a_diamond_room,
make_a_cave_room,
make_a_diamond_room,
make_a_diamond_room,
finish,
...
And the functions are passed the layout and the dungeon, and look like this:
def bottom_left(layout, dungeon):
layout.subregion_LRBT(0, 20, 0, 20)
def make_a_diamond_room(layout: DungeonLayout, dungeon):
if origin := layout.unused_cell_or_none():
room = RoomMaker(layout).diamond(number_of_cells=random.choice([25, 41]), origin=origin)
layout.add_room(room)
def finish(layout, dungeon):
layout.finish_map()
The Stepper has two modes. It can just call all the functions in sequence, all in one go, or it can call them one at a time under keyboard control. The all in one go is for playing the game, and the step by step is for observing the dungeon building process as we configure it.
For anything we we want to do in building up the dungeon, we “just” have to create a new function and put its name in the list. This works as we intended, and I’m sure we felt that it was just a first step toward what we’d ultimately need, but whether we did or not, the current stepper idea has Serious Drawbacks, including but not limited to:
- Even for one-liners like
bottom_leftandfinish, we have to write a function so that the stepper can call it; - There is no provision for printing any kind of ongoing info, other than creating yet another function to print a message;
- There’s no good way to do conditional builds;
- There’s no good way to access information and tuck it away for use in building the dungeon, unless we store info in globals and build yet more functions to save and access it.
So far, in our frequent use of the stepper while working on the path making / subregion idea, I’ve built up a general dislike for how it works. If some poor devils were condemned to working in the dungeon building up things with Stepper, I’m sure I could hear their moans and screams all the way up here in my programming aerie. Anyway, my own irritation with it must have hit some kind of subconscious peak, because
I Have an Idea
The idea has a delightful techie name: coroutines! And Python has a keyword that is just what we need: yield. Let me summarize the ideas for those who need a refresher. Of course we’ll be into the details shortly. Coroutines are used for “cooperative multitasking”. In our use here we’ll be writing a function that can pause in the middle to return a result, and then the next time it’s called, pick up where it left off to provide the next result. The yield keyword is like return, except that on the next call, the function will pick up after the yield instead of at the top.
Python provides yield as part of the notion of “generators”, functions that return a series of result without building the series into a list or other structure, instead just, well, generating values one at a time. And in our use, we’ll use this capability to provide single stepping for our dungeon builders.
For a conceptual example, let’s consider a sequence like the build steps we showed above. If we were not stepping, we might just write a long function that called various other functions:
def build_some_stuff(layout, dungeon):
layout.subregion_LRBT(0, 20, 0, 20)
for _ in range(2):
if origin := layout.unused_cell_or_none():
room = RoomMaker(layout).diamond(number_of_cells=random.choice([25, 41]), origin=origin)
layout.add_room(room)
layout.finish_map()
We would presumably refactor that long function for clarity and convenience, but the point is that it would be written and organized to the liking of the dungeon builder, and relatively unconstrained in what it could do.
But we can’t step a function like that, executing just one bit at a time. Unless we wrote it this way:
def build_some_stuff(layout, dungeon):
layout.subregion_LRBT(0, 20, 0, 20)
yield
for _ in range(2):
if origin := layout.unused_cell_or_none():
room = RoomMaker(layout).diamond(number_of_cells=random.choice([25, 41]), origin=origin)
layout.add_room(room)
yield
layout.finish_map()
yield
Putting in those yield statements turns the function into a Python generator, which means we might call it like this instead of just calling it directly:
for _ in build_some_stuff(layout, dungeon):
pass
That loop would just keep calling build_some_stuff until it stopped yielding things. Since it’s yielding None we have no reason to hold on to them.
But the cool thing is this: When we want to step the build, our stepping code can just call next on the generator, and it will execute from where it left off down to the next yield.
Here are a couple of tests showing the two ways of calling the same generator:
class TestGenerator:
def three_things(self):
yield 1
yield 2
yield 3
def test_three_things(self):
for i, thing in enumerate(self.three_things()):
assert thing == i + 1
def test_three_things_separately(self):
things = self.three_things()
assert next(things) == 1
assert next(things) == 2
assert next(things) == 3
I wrote a bunch of other little tests, putting yields into functions called by other yields and such, which may come in handy later. We’ll look at those if we need them. But this one is germane to the real idea:
def make_room(self, layout):
room = None
if origin := layout.unused_cell_or_none():
room = (RoomMaker(layout)
.diamond(
number_of_cells=random.choice([25, 41]),
origin=origin))
layout.add_room(room)
yield room
def test_cannot_just_call_use_next(self):
layout = DungeonLayout(20, 20)
# must say next here (or loop)
room = next(self.make_room(layout))
assert len(layout.rooms) == 1
assert room in layout.rooms
Here we have a very small bit of code that makes a very small dungeon, and actually yields the room, just so we can see what it does. That said, my present plan is that we won’t yield anything, unless we get some clever idea as we use the new stepper.
OK, more than enough chatter, let’s get to it. I think we should just TDD up a new kind of BuildStepper and plug it in. It’ll be tiny.
class TestYieldStepper:
def test_hook(self):
assert False
Perfect. One test fails. Replace it:
class TestYieldStepper:
def test_create(self):
stepper = YieldStepper()
assert stepper is not None
And, of course …
class YieldStepper:
def __init__(self):
pass
You may be thinking “Why doesn’t he just test something real instead of doing these tiny stupid things”. And yes, I could, and yes, I often do. But I find that starting with these baby steps is easy, requires no particular thinking, and provides a basis where thinking can take place. So now we’ll write tests that are little users of YieldStepper, similar to the real ones that we’ll create Real Soon Now.
def test_run_stepper(self):
sum = 0
def steps():
nonlocal sum
sum += 1
yield
sum += 2
yield
sum += 3
yield
stepper = YieldStepper(steps)
stepper.run_to_completion()
assert sum == 6
run_to_completion is the method on the existing BuildStepper that just executes whatever’s left in the stepper to be done. In the new class it looks like this:
class YieldStepper:
def __init__(self, steps):
self.steps = steps()
def run_to_completion(self):
for _ in self.steps:
pass
Test passes. Let’s commit: working on YieldStepper. Now let’s drive out single-stepping:
def test_maker_single_step(self):
sum = 0
def steps():
nonlocal sum
sum += 1
yield
sum += 2
yield
sum += 3
yield
stepper = YieldStepper(steps)
stepper.maker_single_step()
assert sum == 1
This is enough to fail, although I plan to extend it. maker_single_step is the method we use on the existing stepper, of course, so we’re replicating it here.
def maker_single_step(self):
next(self.steps)
Not terribly difficult. We do have an issue, however, which we’ll talk about in a moment. Let’s extend the test first:
def test_maker_single_step(self):
sum = 0
def steps():
nonlocal sum
sum += 1
yield
sum += 2
yield
sum += 3
yield
stepper = YieldStepper(steps)
stepper.maker_single_step()
assert sum == 1
stepper.maker_single_step()
assert sum == 3
stepper.maker_single_step()
assert sum == 6
Green. Commit.
The issue is this: In the current stepper, we have this:
def run_to_completion(self, view=None):
while self._more_to_do():
self._do_one_step(view)
self._update_view(view)
def maker_single_step(self, view=None):
if self._more_to_do():
self._do_one_step(view)
self._update_view(view)
return self._more_to_do()
I’m not concerned about passing the the view: that’s trivial. I am concerned about _more_to_do. If anyone is using that feature, we have a problem, which is that generators cannot answer whether there is more to do, they can only try. When a generator is called one more time than it has values, it raises a StopIteration exception. In our new run_to_completion we just iterate until done, and the for eats the StopIteration and stops. But in maker_single_step we can’t really replicate the return of True if there is more to do. We can, however, swallow the exception and return something at that point.
In our code for the ‘D’ operation (D for maker single step, obviously), we have this:
elif symbol == arcade.key.D:
more = self.stepper.maker_single_step(self.view)
if not more and not self.called_run:
self.dungeon.run()
self.called_run = True
Basically that’s protection against calling run more than once, but calling it once.
Let’s try this. We’ll require one D past the last yield and do the run then. We’ll return True from maker_single_step if we did a command, and False if we’re off the end.
Write a test for that.
def test_maker_single_step_return(self):
sum = 0
def steps():
nonlocal sum
sum += 1
yield
sum += 2
yield
sum += 3
yield
stepper = YieldStepper(steps)
ok = stepper.maker_single_step()
assert ok is True
assert sum == 1
ok = stepper.maker_single_step()
assert ok is True
assert sum == 3
ok = stepper.maker_single_step()
assert ok is True
assert sum == 6
ok = stepper.maker_single_step()
assert ok is False
assert sum == 6
And:
def maker_single_step(self):
try:
next(self.steps)
return True
except StopIteration:
return False
Green. Commit.
Reflection
This is an excellent opportunity to reflect, and, quite likely, to call a break in the proceedings, since the object is done (except for the View parameter, which is still trivial) needing only to be put into production and fed a sequence for its debut and final testing.
The existing BuildStepper is 33 lines long and is not as capable as this one is, while also being harder to use. The new one is presently 14 lines and will add fewer than ten when we add in the view updating which it does in fact need to do. The real proof of the idea, of course, will come when we create a sequence and actually use it to build a dungeon, testing it all in one go, and step by step (slowly I turned …).
I have one semi-large concern. It’s only one line, how large could it be, but it is this:
class YieldStepper:
def __init__(self, steps):
self.steps = steps()
YieldStepper calls the steps generator. What will happen if we need a parameterized generator? Let’s try to test that before the break.
def test_parameterized(self):
sum = 0
def steps(count):
nonlocal sum
for c in range(count):
sum += (c + 1)
yield
stepper = YieldStepper(steps(3))
stepper.run_to_completion()
assert sum == 6
This fails trying to call the already called generator:
E TypeError: 'generator' object is not callable
OK, new rule, pass in a called generator. Or we could get fancy and call it if it needs calling? No let’s not.
class YieldStepper:
def __init__(self, steps):
self.steps = steps
Commit that. Let’s sum up and break.
Summary
This is going nicely. The yield works perfectly, and I’m sure we’ll be able to put as many yields in as we wish, keep track of things inside the build, and so on.
It’s tempting to push on and finish up, just need to put in the view and its update, which we can copy from BuildStepper, and then create an example. But I’ve been coding and writing for a couple of hours, so taking a break seems prudent. See you later!