FAFO on GitHub

Let’s try an experiment, along the lines of my sketch at the end of this morning’s article. It goes very well. Very well indeed!

Hello, friends!

In this morning’s article, I wishfully sketched this idea:

for dept in personnel.grouped_by('department')
    print(dept.name)
    for job in dept.values.grouped_by('job')
        print("    ", job.name)
        for rec in job.values.sorted_by('job')
            print("        ", rec.name, rec.pay)

This afternoon, if the interruptions stay low enough, I propose to try to implement something like that. Let’s write a small test:

    def test_group_by(self):
        personnel = self.build_peeps()
        for dept in personnel.group_by('department'):
            print(dept.name)

I envision the group_by operation returning a set of objects, with two members, one a value, ‘name’, and the other, as we’ll see later, a set. Let’s try to do this much.

So far so good:

    def group_by(self, a_scope):
        projector = XSet.classical_set((a_scope,))
        projected = self.project(projector)
        group = []
        for e, s in projected:
            name = e[a_scope]
            obj = GroupHolder(name, None)
            group.append(obj)
        return sorted(group, key=lambda x: x.name)

class GroupHolder:
    def __init__(self, name, values):
        self.name = name
        self.values = values

The test prints:

it
sales

Not impressive but just what we wanted.

Test is green. Should we commit? Sure: starting on group_by method in XSet.

Let’s extend the test to drive out the values.

    def test_group_by(self):
        print()
        personnel = self.build_peeps()
        for dept in personnel.group_by('department'):
            print(dept.name)
            dept_values = dept.values
            for job in dept_values.group_by('job'):
                print("    ", job.name)
        assert False

This fails for there being nothing to iterate, as values is presently None. We revise group_by:

    def group_by(self, a_scope):
        projector = XSet.classical_set((a_scope,))
        projected = self.project(projector)
        group = []
        for e, s in projected:
            name = e[a_scope]
            selector_rec = XSet.from_tuples([(name, a_scope)])
            selector_set = XSet.classical_set([selector_rec])
            values = self.restrict(selector_set)
            obj = GroupHolder(name, values)
            group.append(obj)
        return sorted(group, key=lambda x: x.name)

And we get this report:

it
     sdet
     serf
sales
     closer
     prospector

I call that nearly good. Commit: progress on group+by.

I wonder whether select would be easier to manage than restrict.

Best idea yet today:

    def group_by(self, a_scope):
        projector = XSet.classical_set((a_scope,))
        projected = self.project(projector)
        group = []
        for e, s in projected:
            name = e[a_scope]
            values = self.select(lambda detail, _s: detail[a_scope] == name)
            obj = GroupHolder(name, values)
            group.append(obj)
        return sorted(group, key=lambda x: x.name)

Misses out that complicated double set thing, and probably a smidge faster as well. Commit: use select instead of restrict.

Let’s advance the test further:

    def test_group_by(self):
        print()
        personnel = self.build_peeps()
        for dept in personnel.group_by('department'):
            print(dept.name)
            for job in dept.values.group_by('job'):
                print("    ", job.name)
                for worker, scope in job.values:
                    print("        ", worker['pay'])
        assert False

This nearly does what we want, but not quite:

it
     sdet
         11000
         10000
     serf
         1000
         1100
sales
     closer
         1000
         1100
     prospector
         10000
         11000

The detail records are not sorted. We could do this:

    def test_group_by(self):
        print()
        personnel = self.build_peeps()
        for dept in personnel.group_by('department'):
            print(dept.name)
            for job in dept.values.group_by('job'):
                print("    ", job.name)
                for pay in sorted([worker['pay'] for worker, scope in job.values]):
                    print("        ", pay)
        assert False

That gives the desired result in this case:

it
     sdet
         10000
         11000
     serf
         1000
         1100
sales
     closer
         1000
         1100
     prospector
         10000
         11000

Commit: Report coming out as desired. Promising!

I think that’ll do for now. We have an early supper planned. Let’s sum up.

Summary

I fantasized about what I wished I could write to produce a report, and then made the fantasy come true with a small object and a small group_by method in XSet. Much simpler than the XGroup idea, requires no new implementation type, and is substantially easier to use.

There is a lesson here. When something seems too difficult or awkward, we can look for a different path to the desired result.

Another lesson, quite probably, is something like “if reporting is your problem, start with reporting”. Starting with an assumed solution deep in the system was a dead end.

Should I feel badly about that? Not really. There’s no telling what would have happened had I gone in some other direction. Little time was lost, we learned things, and we have a nice result coming into being now.

Small steps. Not every one will be just right. Sometimes we abandon an idea, and we may well abandon the XGroup idea. But what is this new GroupHolder object, really? What is its true name and true purpose? We surely do not know its name, and we may not know its purpose as yet.

Maybe, just maybe, it’s a set with interesting properties and we just haven’t noticed yet. Maybe … it’s a nascent XGroup?

We’ll find out as we move forward in more small steps.

See you then!