Prototyping and Polyscheme
Saturday, March 15, 2014
I’ve been working on a personal project I call PolyScheme which is a variation of Scheme with first-class types. I’ll talk about the results when it’s a bit farther along, for now I want to talk about how I created the project using prototyping, an approach to software development where you first create a working prototype using rapid prototyping techniques and once you’ve got it working rewrite it in a more performant (and management-approved) language.
This is hardly a new idea. I spent quite a bit of time trying sell the use of Smalltalk for building prototypes in the late 1980’s and early 1990’s with C++ or (shudder) Cobol as the planned final implementation language. With very few exceptions I and my clients never made this work. Failure came in two forms, the Too Expensive failure usually involved management saying something like “Why would I want to pay for this to be developed twice?” The Ship It failure usually involved words like “What do you mean rewrite it? It works, just ship it!” (Which if you think about it, isn’t much of a failure at all).
With the exception of a few toys, my personal projects have suffered from the same thinking. Typically I do a preemptive Ship It and write the code in whatever will make the task as quick and painless as possible with no thought to rewriting. For projects whose final form needs to be in a more traditional language (usually Java) and in my more professional work I tend to subconsciously use the Too Expensive excuse.
Honestly I would have done the same with PolyScheme, which I planned to write in Java so I could leverage it as an application scripting language. I didn’t because of Google. I’m just coming off a two-and-a-half year stint as a software engineer at Google Boulder a job that was sufficiently demanding that I dropped pretty much all of my personal projects. PolyScheme and a bit of fiction writing were all that I managed to squeeze in during this time, and the thought of Java development in Eclipse outside of work time was just not something I could handle. Instead I decided to work out some of the tricky parts in Lua and ended up writing a working if somewhat limited version of PolyScheme. Now that I’ve left Google and have more time, I’ve started rewriting PolyScheme in Java. To my shock—and amusement at my own shock—the rewrite has gone incredibly smoothly. The only rough spot was the code that integrates with Java which by necessity was not part of the Lua version.
Prototyping works. I knew that but haven’t thought about it in far too many years.
I think prototyping works for a very simple reason. When we’re writing code we’re thinking about two things, the problem we’re solving and writing code. Each of these is a moderate to very difficult task and combining them is one of the reason programming is hard. Prototyping breaks the task up into two parts. In the first part the programming task is made as easy as possible by using languages, libraries and systems that are easy to program in which allows the developer to focus on the problem. In the second phase the heavy thinking about the problem is done and the developer can focus on writing code.
There’s an important lesson hidden here, a reason why developers might support management in assuming prototyping is Too Expensive. Writing a prototype in an unfamiliar prototyping language adds a third thing to think about which can be summarized as “How do I do X in this explicative deleted language?” Thus there’s no immediate benefit to using an unfamiliar prototyping language over a familiar production language. It’s only after the prototyping language has been mastered that developers can see the benefits.