Let Over Lambda - An Object System in Python

Let Over Lambda, meaning a let-clause (binding variables) over a lambda (anonymous function) definition, is a programming pattern frequently used in Lisp utilizing a lexical closure to, essentially, define something akin to an object: One or more functions that close over an environment.

Let’s do it in Python. Python 3 supports closures using the nonlocal keyword:

def closure():
    """Return lexical closure over 'x'."""
    x = 0

    # The function that increments x.
    def fn():
        # Closure over 'x'.
        nonlocal x
        x += 21
        return x

    # Here we return 'fn', which closes over 'x'.
    return fn

We get a fresh instance by calling closure(), which we can use to successively add 21 to the closed-over x:

f = closure()
f()
assert f() == 42

Indeed, the function closure() is the pythonic version of a Lambda Over Let Over Lambda, that is, a function returning a fresh closure – or a constructor, in object-oriented terms. Of course, in Python we usually use the class keyword to define classes. Let us consider the traditional example of a simple class Counter, which we might define as follows.

class Counter0:
    """A counter for counting."""

    def __init__(self, init=0):
        """Intantiate a new counter."""
        self._x = init

    def inc(self, by=1):
        """Increment counter."""
        self._x += by
        return self._x

    def dec(self, by=1):
        """Decrement counter."""
        self._x -= by
        return self._x

c0 = Counter0(40)
c0.inc(3)
assert c0.dec() == 42

How could we represent such a counter object without using class? We already know how to encode the internal state, using closures. Our constructor function can return two functions inc and dec, instead of a single one. Of course, it can take an argument for initialization as well. We are thus quickly approaching a first version of our alternative object system.

def Counter1(init=0):
    """Instantiate a new counter."""
    x = init

    # Increment the counter.
    def inc(by=1):
        nonlocal x
        x += by
        return x

    # Decrement the counter.
    def dec(by=1):
        nonlocal x
        x -= by
        return x

    # Return the two 'methods'.
    return inc, dec

c1_inc, c1_dec = Counter1(40)
c1_inc(3)
assert c1_dec() == 42

Unfortunately, we need to bind n variables for objects with n methods, for each instance, and referring to both the object (c1 in this case) as well as the method we want to call (inc or dec). This will probably become rather unwieldly, rather quickly. We can improve upon this version, if we instead return a datastructure (e.g., a map) such that we need only one variable per object, and can access methods by their defined names.

def Counter2(init=0):
    """Intantiate a new counter."""
    # ...

    # Return a dictionary containing all methods.
    return {
        'inc': inc,
        'dec': dec,
    }

c2 = Counter2(40)
c2['inc'](3)
assert c2['dec']() == 42

Well. I mean, it certainly works, and we get to call our methods in a slightly more sane way, but the calling syntax is still a bit too wild. Perhaps we can return a dispatch function, that looks for the correct method we want to call?

def Counter3(init=0):
    """Intantiate a new counter."""
    # ...

    # Dispatch on supplied message, forwarding other arguments.
    def dispatch(msg, *args, **kwargs):
        match msg:
            case 'inc':
                return inc(*args, **kwargs)
            case 'dec':
                return dec(*args, **kwargs)
    return dispatch

c3 = Counter3(40)
c3('inc', 3)
assert c3('dec') == 42

It does save us the extra parentheses. Indeed, this is somewhat similar to a solution in Lisp. However, we can note that while this patterns does make sense in Lisp’s syntax (where we would call something like (c3 :inc 3), in Python it is still a bit weird. Can be make this look more like a normal Python object? We can use a SimpleNamespace from the standard library, which is, essentially, a glorified dictionary with . syntax.

def Counter4(init=0):
    """Intantiate a new counter."""
    # ...
    from types import SimpleNamespace

    # Return a SimpleNamespace, so methods can be called with '.'.
    return SimpleNamespace(**{
         'inc': inc,
         'dec': dec,
    })

c = Counter4(40)
c.inc(3)
assert c.dec() == 42

Great! A complicated way of defining classes. But we can do even better by reducing some of that boilerplate. To this end, we could define a helper function that constructs the class from functions in locals(). Thus, we only need to call this helper function and return its result in order to turn our closure into a class.

def cls():
    """Make a new 'class' instance by using 'return cls()' in a closure."""
    from types import SimpleNamespace, FunctionType
    import inspect

    frame = inspect.currentframe().f_back
    localz  = frame.f_locals
    del frame
    return SimpleNamespace(**{
        name: fn for (name, fn) in localz.items()
        if isinstance(fn, FunctionType)
    })


def Counter5(init=0):
    """Intantiate a new counter."""
    # ...

    # Return a class from local functions.
    return cls()

c = Counter5(40)
c.inc(3)
assert c.dec() == 42

Arguably, this version even has some actual benefits over normal Python classes: The variable x is actually private (i.e., not accessible without too much trickery) and one can easily imagine extending the cls function to omit some functions marked as private, too. It is also, maybe surprisingly, less verbose than regular Python, except for the explicit naming of nonlocal variables in each method. (Which, one could say, makes access and possible mutation of members explicit and might be considered a feature.) Finally, we are encouraged to use composition over inheritance (given that the latter is not possible).

Finally, here is a variant of the previous version, using a decorator to build the namespace without using inspect. This version does require explicitly lising all public methods in the return clause. This is a little bit of overhead over the previous version, but it allows easily omitting some private methods, and thus makes the public interface explicit.

def cls(constructor):
    """Decorator to make classes from closures."""

    from types import SimpleNamespace
    from collections.abc import Iterable

    def make_class(*args, **kwargs):
        methods = constructor(*args, **kwargs)
        # 'methods' might be a single value.
        if not isinstance(methods, Iterable):
            methods = (methods,)
        return SimpleNamespace(**{
            f.__name__: f for f in methods
        })
    return make_class


@cls
def Counter6(init=0):
    """Intantiate a new counter."""
    # ...

    # Return a class from local functions.
    return inc, dec

c = Counter6(40)
c.inc(3)
assert c.dec() == 42

Now, at this point I was tempted to implement inheritance, which kind-of worked by giving the cls decorator an optional argument, a superclass, and fiddeling with the namespaces. But since this became quite ugly and probably missed about a thousand edge-cases, I leave this as an exercise of the reader. The full version of counter 0 through 6 are available for download here.