The Dongola Times

(Anachronistic) Dispatches from the Kingdom of Makuria.
05th of June, 2013

Re: Frameworks are Futamura Projections

In practice, I have found that achieving the second projection means a lot of using cp and vi. (This is if I assume that, in my work, I have implemented the projections; such that I am just looking to draw a corollary from my work to the Futamura projections.) So if the programmer is part of the specialiser, then copying and modifying of a compiler is a process that is achieved by a compiler-compiler (and this includes the programmer in its definition); it is a process achieved by a specialiser. The hard-math specialisers on the market are trying to excise the human from the picture; but then you end up with merely a compiler (a simple specialiser), because the only way to create a recursive system is by making a programmer part of it. A true specialiser à la Futamura, alas, requires that there be a human as part of the specialiser. I think my next proof should be for that; and a good one (I hope). If true, this establishes that just as observation is fundamental in quantum physics, observation is fundamental in optimisation.

At least that’s how it looks from my own projects, if I look for something like the Futamura projections in them. The mental acrobatics required to see every framework as the application itself (just wanting for some arbitrary parameters), and every application as the framework (just wanting for some arbitrary parameters) is something programmers do every day. Whenever you see your earlier work and notice that you could just copy code from here and change this and that, you have executed the mental acrobatics and even brought forth one of the specialisers in Futamura’s projections. If I am right, our minds would be doing this very fast subconsciously, matching patterns. These NP-hard things seem to go very fast in the subconscious with what we think are probabilistic algorithms. We assume they are probabilistic algorithms we use to make these quick comparisons, because a linear, non-intuitive judgement could rule against their confident answers; but when you think it through, you may find that you match patterns in a different way. Our identification of the projections, and what parameters to feed to what to generate what—what we should copy, what we should modify, and with this our head-machine churn out a new framework-application that we can copy again later—is a pattern-match; a very fast one, at that, if indeed we, as programmers, routinely manage to condense the Futamura projections into invocations of cp and vi.

But this leaves me in the very paradoxical spot where “don’t repeat yourself”, which the projections of Futamura allow us to hope for, is only ever attained by copying again and again, and repeating ourselves again and again. For vi introduces the new parameter, while cp feeds the machine into itself. —Because cp makes the old work “new” to the same programmer, and now allows vi to bring to bear new parameters.

The weight of cp in all this is very important, because file-copying happens also to be the one of biggest jobs of a framework, such as Ruby on Rails, Django, or GAE. With Rails, we have a command to basically copy a billion crazy files into an agreed location; it is the first thing you do in Rails. Django generates fewer files, but since it also has file system convention, the same charges hold. GAE is rather worse; it requires a precise conf file (app.yaml or sometimes supposed to be called app.yml, depending on the language!) which it never helps create, and it also requires that you cp all your libraries. Their “specialiser” at GAE doesn’t know very much about libraries, so you have to think much more. (In other words, where Rails is a Sufficiently-Smart Specialiser, GAE is crap.) And as I have said, for the programmer here, the projections are done with cp, so he copies the libraries over to the GAE directory. And GAE copies these over to the specifier at Google. (Note that they have their own interpreter which they have deliberately disabled in parts, which is why the input is lower-level, and GAE is, similar to Django, quite light-weight.)

And to be cheeky, let me say that the third projection of Futamura may be sex between a programmer-type guy and programmer-type girl. Or maybe He just misses Her. ;o)

In one of the more-blatant exploitations of pure mathematics to make my life easier, I am henceforth going to be seeing web frameworks as Futamura Projections, and this means that I will use the Ruby on Rails ideology of “don’t repeat yourself”. Since every framework is actually the application (minus just a few parameters), every application I build will be a framework (given some parameters), and every framework I build will be an application (leaving room for parameters, of course).

And this is, in fact, the difference between libraries and frameworks. Libraries are like viruses in biology; there is a lot of horizontal gene transfer when you use a library, while frameworks are like sex within a species, where the parents are continued in the children, with a few parameters tweaked. If a stupid biologist—almost the only kind of biologist in the mainstream today—were to look at the apps on a computer, he would think they all arose from libc.a perhaps by random mutation or whatever; after all, all of libc is contained in all of the programs on the computer! “Yay! No need for the Programmer!” Clever can be wrong (the forgetting of which is responsible for our highest stupidities).

So, less talk, more type.