A Day In The Lyf

The Sapir-Whorf hypothesis is a famous linguistic theory that postulates that what you think is constrained by what you can say. In other words, the way you understand the world is limited by the language(s) you speak.

The hypothesis is regarded as incorrect by modern linguists. However, the spirit of the theory appears to be largely true if we refer only to computer languages – our solution space is limited to the number of ways we know to express solutions. Paul Graham was indirectly talking about Sapir-Whorf when he described the Blub Paradox. Blub, the language that sits halfway up the hypothetical totem pole that represents the relative power of computer languages, has certain constructs that make languages lower on the pole seem downright dysfunctional. The Blub programmer would look down, realize that the language he’s looking at doesn’t even have some language feature X, and scoff at how much extra work it would be to write programs without X. X, the Blub programmer argues, is so fundamental to designing your application that it would criminal to use any languages without X.

When the Blub programmer looks up the totem pole, though, he just sees weird languages. What he doesn’t realize, of course, is that programmers in those languages are looking down at Blub and wondering how anybody could possibly program without language feature Y.

The Blub paradox makes a nice story, and while it has value, it oversimplifies the language landscape. There isn’t a linear totem pole for expressive power when it comes to languages. Graham suggests looking at the size of the parse tree, which is indeed a useful single metric, but ignores aspects like long-term maintainability, readability, syntactic sugar, etc. The computer language landscape is diverse precisely because there are different opinions on the best way to solve different problems.

What should be obvious is that not all software problems should be solved with the same tool. Too many programmers go down that path, and use Java (or C#, or C++...) as the hammer and let everything become a nail.

Granted, we don’t always have control over which language we use. We write programs in Java (or C#, or C++...) because that’s our job, even though we may suspect that it’s not always the best tool. Fair enough. But if our ability to describe solutions to a problem is constrained by the programming languages we know, doesn’t it make sense to learn other languages even if we can’t use them on the job?

Knowing a diverse set of programming paradigms can help you become a better programmer. It’s helped me. For example, I wrote a closure-based undo in C#. A more typical OO-based undo would have resorted to the command design pattern, but the closure-based implementation required much fewer changes to the codebase. I don’t think I would have come up with the solution if I had not had some prior exposure to functional languages.

The key is to make sure that when you pick up a new language, that it’s different enough from languages you already know that you force yourself to wrap your head around some new concept. You don’t get that, for instance, going from Java to C#. I’ve tried learning at least a little about several languages, and found useful new concepts in the following:

• C# / Java (statically typed OO)
• Eiffel (design-by-contract, anchored types)
• Ruby (open classes, incredible reflection, metaprogramming)
• Smalltalk (like Ruby, but with keyword arguments and a very lightweight syntax)

• Perl / Python (dynamic typing, default variables, list comprehensions)
• C++ / Delphi (destructors)
• JavaScript (prototypes instead of classes, hybrid functional)

• C (weak typing, systems level procedural, teaches good understanding of computer memory)

• Scheme / Common Lisp (homoiconic, syntactic macros, multiple dispatch, continuations, no penalty for tail recursion)
• Erlang (nice distributed model, pattern matching)

• shell scripting (using processes as the unit of abstraction)
• SQL (yes, even the much-maligned relational, set-based model provides another, occasionally superior, way of solving problems)
• XSLT (declarative)

• Prolog (propositional logic programming)
• Forth (stack-based programming)

PragDave recommends that you learn a new programming language each year. It’s terrific advice. Pick a language, find some coding exercises, and try to wrap your head around solving those problems idiomatically in the new language. Don’t be surprised if you start out solving them the same way you would have in a language you already knew, but remember the end goal is to learn new problem solving techniques, not just new syntax. Since you probably won’t be using the language on the job, you’ll almost certainly forget the syntax after a while. Start posting questions to mailing lists, blog about some of your sample work, and don’t be afraid to make a fool of yourself. The way I see it, if you’re still afraid to make a fool of yourself, then you clearly haven’t made a fool of yourself enough.

My problem is that I tend to read a book or two on a language, maybe try it out with an academic exercise, and move on. Impatience may be one of the three virtues of all great programmers, but I don’t think that’s the kind of impatience Larry Wall had in mind. So I’ll try to take my own advice, and start periodically blogging some sample small projects in languages that I don’t get to use on the job.

I’m hoping to find projects that are small enough that I can do them in my (very limited) free time, but still big enough to teach me something about the language. I’ve seen too many bowling examples that have very little to do with object-orientation.

Stay tuned…

Update: A few short hours after posting this article, Ola Bini left a comment explaining how calling what Ruby does ‘parse-time execution’ is just plain wrong. In hindsight, it seems like a silly mistake to make. Of course Ruby parses a file before executing it – how many times had I seen the standard parse error? Ah well, that’s why we blog, no? Making silly mistakes is all part of the learning process.

Rather than try to hide my idiocy, I decided instead to clarify – to myself as much as anyone – what my thinking was that led me to the mistake. I had just learned about syntactic macros in Lisp, and was a bit overzealous in applying the same concepts to Ruby. However, I do believe that Ruby provides many of the same benefits. Being able to include things that are typically syntactic definitions as part of the execution environment makes it less important whether the code executes at parse time or execution time. Ruby provides that ability. If you just ignore the silly ‘parse-time’ phrase, and focus instead on the fact that we are defining syntactic constructs at execution time, then the advantage becomes clear.

End Update

My recent foray into understanding Lisp macros got me thinking more about code that executes at parse-time. While few other languages have access to the raw parse tree, many other languages have parse-time executable code. Having worked in some of those languages, I’d seen it before, but never really thought about the difference between parse-time and run-time execution until recently.

Developers tend to think of executable code as a run-time concept. This is especially true in the mainstream static languages of the day. Indeed, the very term “run-time” acts as a conceptual block for many of us (including, until recently, me), for it is intended to describe the environment in which code can run. What is ignored in that concept is that interpreted languages can execute code as it’s parsed, before the entire application has finished parsing, and some languages let you execute code as the source code is compiling.

In most cases, the difference isn’t worth thinking about:

puts "Hello, world!"
puts "Goodbye!"

The Ruby code above shows a trivial example. The first line of code is parsed and executed. “Hello, world!” prints on our console before the second line is even parsed. You can verify this simply by adding a syntactic error on the second line:

puts "Hello, world!"
blarf "Goodbye!"

As you may suspect, blarf isn’t a predefined function in the Ruby language, and when the second line executes, the Ruby interpreter will spit out a NoMethodError to us. However, it will not do so until first saying hi to us! The first line parses just fine, and before parsing the second line, it is executed.

OK, so what? The example above isn’t very interesting. Let’s show another example. Ruby allows open classes, which means that you should be able to change a class even after it’s been defined:

class Dog
  def wag_tail
    puts "tail wagged..."
  end
end

class Dog
  def bark
    puts "Wuff!"
  end
end

So now a Dog can both wag it’s tail and bark. But notice what happens if we get Evil (which basically means we start thinking like Microsoft or Sun), and want to seal our class:

class Dog
  def wag_tail
    puts "tail wagged..."
  end
end

Dog.freeze

class Dog
  def bark
    puts "Wuff!"
  end
end

The only difference between this code and the code above is the call to freeze, which makes the receiver immutable. The receiver in this case is the Dog class, and so when we get to the next line, which tries to reopen the class, the interpreter throws a TypeError. So a line of code that has already executed causes an exception to be thrown when the next line is parsed.

That’s a little bit more interesting, but not very instructive unless you want to be a framework developer for Microsoft or Sun. Let’s find a better example; take a look at the following code:

class Blog
  def title
    @title
  end

  def title=(value)
    @title = value
  end
end

This code still gets executed as it’s parsed, but all it does is add the Blog class to the symbol table. The getter and setter methods are added to the class, but the code within those methods isn’t executed.

Of course, no self-respecting Ruby programmer would write the code above. Instead, it’d look like this:

class Blog
  attr_accessor :title
end

Once the Ruby interpreter gets to the end keyword, it knows it has parsed a complete executable instruction and executes it. Again, the Blog class is added to the symbol table. But we’re no longer statically adding methods to the class. Instead, attr_accessor, a method of Class, is executed. attr_accessor adds the getter and setter to the class when it gets executed (essentially by eval‘ing the boilerplate code above). We depend upon attr_accessor running at parse time! Otherwise, our getter and setter wouldn’t exist.

Parse-time executable statements are often called macros in Ruby, which denotes their similarity to Lisp macros (although Ruby lacks access to the parse tree). Rails has become popular in part for its ability to use macros to simplify your configuration:

class Blog < ActiveRecord::Base
  has_many :articles
end

Here we’re managing our object-relational mapping relationships using a macro. As the has_many statement gets parsed, the various methods to manage the relationship get added to the Blog class.

When you first grok macros, you start seeing duplication that you never would have noticed before:

class Rollback < ActiveRecordError
end

class DangerousAttributeError < ActiveRecordError
end

class MissingAttributeError < NoMethodError
end

The code above was stolen from ActiveRecord::Base. We could, if we choose, eliminate the duplication with something like this:

  expose_exceptions ActiveRecordError, :Rollback, :DangerousAttributeError
  expose_exceptions NoMethodError, :MissingAttribute

expose_exceptions, as shown here, takes the exception’s superclass as the first argument, followed by a list of exception class names. It would be trivial to implement, but it’s not the approach Rails takes, and for good reason. While there is indeed duplication in the Rails code, it is justifiable, since it allows a body of comments (stripped out in my example above) to explain what the exception is there for.

What else does parse-time execution get us? On a previous project, we used Rails fixtures to store our test data, even though we weren’t using Rails (I wrote about this here). Rails fixtures were designed for testing, but because they facilitate a fairly nice way of storing test data (in YAML), we co-opted it for that purpose. To make it work, we had to write an ActiveRecord class for each of our tables. It was a pretty mechanical process, but we had to override certain things since we didn’t abide by Rails naming conventions (and what conventions we claimed to abide by were applied inconsistently). The following definitions were typical:

class Account < ActiveRecord::Base
  set_table_name 'Accounts'
  set_primary_key 'AccountId'
end

class Order < ActiveRecord::Base
  set_table_name 'Orders'
  set_primary_key 'Id'
end

After several of these definitions, the duplication became obvious. We couldn’t completely eliminate it because of our own naming inconsistencies, but at least we could make reasonable defaults that could be overridden if needed. In particular, notice the table names are the plural of the class names (Rails expects this too, but with lower cases and underscores). set_table_name is a class method, not an instance method, so it may not be immediately clear how to eliminate that duplication.

Ruby provides certain hook methods during parse-time events. One such hook is called when your class is subclassed. We used that to trigger a call to set_table_name:

class StandardTable < ActiveRecord::Base
  set_primary_key 'Id'

  def self.inherited(subclass)
    subclass.class_eval "set_table_name '#{subclass.name.pluralize}'"
    super
  end
end

class Account < StandardTable
  set_primary_key 'AccountId'
end

class Order < StandardTable; end

As soon as a command to subclass StandardTable is parsed (which means reaching the end statement), the class level inherited method hook is invoked, which eval’s the default set_table_name. As Account shows with set_primary_key, the subclass can still override the macros if needed.

Executing code at parse-time is an extremely powerful technique, even without access to the parse tree. Languages that don’t allow you the option of parse-time execution weaken your ability to make powerful abstractions. Working in languages that allow you parse time execution expand your thinking in useful directions, allowing you to spot duplication that you might never have seen before.