andymatuschak.org: Square Signals

So I was sitting in a lounge full of freshmen, typing furiously, when suddenly I shouted with glee:

“My network! It neurals!”

My task, and I had to accept it:

I’m taking this course called “Learning Systems,” which teaches things like neural networks, genetic algorithms, and other buzzword-sounding devices. Our first assignment went along the lines of:

“Design and implement a neural network that learns through gradient descent on its error function. We give you a bunch of training data; you give us a function that generates outputs for all inputs with an error of less than 10^-3 and has low probability of being a local minima. You’ve got one week—go.”

That wouldn’t be so bad, except that this is only one of my five classes, and I like sleeping. Eep.

Traitor!

So I need to do this fast. You might think that as a zealous Cocoa programmer, I started up Xcode and started writing a bunch of @interface declarations. Nay, I say!

This summer, I wrote my research project with Ruby on Rails; this gave me a huge appreciation for that spunky little language. Much as I like Cocoa, I’ve found that since I’ve returned to Objective-C this fall, there’s a whole lot I miss in Ruby.

Some data with a dash of hearsay

Other students in the course fall pretty cleanly into the categories of “computer scientist” and “programmer,” so the other languages used were pretty much exclusively Haskell and Java for the respective groups.

I talked to my friends afterwards to see how things went. Haskell isn’t really suited to a problem like a neural network simply because of the sheer amount of state that has to be maintained throughout the problem. Many of those using Java succeeded, but implementations ranged from 700 to 2,000 lines of code. Those using C had similar results.

My mind boggles: using Ruby, I got a working neural network in about 60 lines of code, plus 30 more to read the test data and to analyze the results.

A simple lines-of-code comparison is a dumb way to compare two languages; what’s important is really the required effort. But based on my conversations with classmates, it took me a lot less time, too. If you think I only did it faster because I’m a better programmer, you’re crazy: this is Caltech, and I’m really nothing special here.

My argument here is that Objective-C could be a lot better—and 2.0 a lot more useful—if it had taken some cues from really agile languages like Ruby, Python, and Smalltalk. I’ll take you through some of the niceties.

The difference

So what’s the difference? What can be learned here? It’s hard to distill exactly what makes Ruby such a ridiculously fast prototyping language, but I’ll try to provide some code samples to make things clear.

We’ll start with something simple. Ruby built-in looping abilities are actually quite simplistic. It has loop (which just runs the code inside until break is called) and while. And until, which is the very useful opposite of while. But on a day-to-day basis, you want to do something like this:

for (int i = 0; i < 10; i++) { array[i] = i; }

In Ruby, that looks like this:

10.times { |i| array[i] = i }

Not too different, not too much shorter. Not worth switch for, anyway. Conceptually, it’s simpler, but you’re used to typing for loops. What about something more complicated?

for (int i = 10; min <= 20; min++) { NSLog("%d\n", i) }

In Ruby, that looks like this:

10.upto(20) { |i| puts i }

How many numbers are printed? An off-by-one error in this kind of thing is easy. Not in Ruby: 10.upto(20) is pretty clear. The whole language has a knack for this kind of thing: when I’m using Rails, I can type things like 1.hour.ago to get the time an hour ago.

Warning: Math!

More complicated examples are in order. Here’s one formula I needed to implement to determine the error gradient (don’t worry about what things mean):

This is clearly recursive, so what’s the base case? Well, for the last layer, the sum changes into a constant value, but the coefficient stays the same. I don’t want to have to rewrite that, of course. Here’s my Ruby implementation (# starts a comment):

delta = (1 - (output**2)) * if next_layer.nil?
  2 * (output - expected_value) # Replacement for the sum in the base case
else
  next_layer.neurons.inject(0) { |sum, n| sum + (n.weights[index] * n.delta) }
end

And here’s the tightest implementation I can come up with in Obj-C 2.0:

float delta = (1 - pow([self output], 2));
if ([self nextLayer] == nil)
  delta *= ([self output] - [self expectedValue]) // Base case value
else
{
  float sum = 0;
  for (id neuron in [nextLayer neurons])
    sum += [[neuron weights] objectAtIndex:[self neuronIndex]] * [neuron delta];
  delta *= sum;
}

Besides just being more verbose, there’s a few things to notice here. In order to keep the code as trim as possible in the Obj-C version, I’ve set the value of delta to some preliminary value, then I *= it by the second part to get the final value. This is okay for now, but if I come back later and add more parts to the equation, I might get confused: delta isn’t actually the real delta value initially. In the Ruby version, however, I’m able to just multiply by the result of the if statement. So there’s one interesting feature of Ruby: everything returns some value (even if it’s just nil).

A simple thing to notice is how I access a particular weight in Ruby: n.weights[index]. Arrays are used all the freaking time. For everything you do. Language-level support for them is a useful thing.

The real piece of magic I’ve included here is inject.

enum.inject(base) {| memo, obj | block } => obj
enum.inject {| memo, obj | block } => obj

Combines the elements of enum by applying the block to an accumulator value (memo) and each element in turn. At each step, memo is set to the value returned by the block. The first form lets you supply an initial value for memo. The second form uses the first element of the collection as a the initial value (and skips that element while iterating).

# Sum some numbers
(5..10).inject {|sum, n| sum + n }          #=> 45
# Multiply some numbers
(5..10).inject(1) {|product, n| product * n }
    #=> 151200
 
# find the longest word
longest = %w{ cat sheep bear }.inject do |memo,word|
  memo.length > word.length ? memo : word
end
longest                                     #=> "sheep"
 
# find the length of the longest word
longest = %w{ cat sheep bear }.inject(0) do |memo,word|
  memo >= word.length ? memo : word.length
end
longest                                     #=> 5

This is so useful—I can’t even begin to tell you. But the Ruby concept that really makes this feasible is blocks, which are (usually) inline subroutines that can be passed around, preserving the scope of the parent function.

I freaking love blocks. Seriously.

How many of you have tried to find out if some element of an array has some property? It’s a pretty common idiom:

NSArray *array = [NSArray arrayWithObjects:[NSNumber numberWithInteger:4],
                                           [NSNumber numberWihtInteger:5],
                                           [NSNumber numberWithInteger:6],nil];
BOOL found = NO;
for (id item in array)
{
  if ([item doesSomething])
    found = YES;
}
return found;

This sucks. Here’s something better:

[4,5,6].any? { |item| item.doesSomething }

There’s also .all? and .each and so on. I’m not saying Ruby is awesome because all these methods exist. Though they are very cool. The point is that on the rare occasion when something doens’t exist, you can write it with blocks. Look at sortedArrayUsingFunction:context: in Obj-C. To do anything with it, you have to go write a (hard-to-name) method separately and pass it along. But you’re only using that method you’re passing in one place; it should be defined as you pass it. Say I want to sort words alphabetically by the last letter. Here’s how that works in Ruby:

["hamster", "cat", "mouse"].sort { |a,b| a[-1] <=> b[-1] }

Note that an array index of -1 returns the last element, -2 the second to last, and so on. This simple case can actually become:

["hamster", "cat", "mouse"].sort_by { |word| word[-1] }

Doing this in Obj-C would have been a huge hassle. You would have had to go and write some method that would compare the last letters of two strings. And what would you even call it? compareByLastLetterTo:?

Hashes for semantic, readable code

I didn’t realize until this summer just how incredibly useful dictionaries (aka hashes) were. I think that’s mainly just because they’re such a freaking huge hassle to work with in Obj-C. See how much work it is to make and look through a dictionary:

NSDictionary *dict = [NSDictionary dictionaryWithObjectsAndKeys:[NSNumber numberWithInteger:4], @"cats",
                                                                [NSNumber numberWithInteger:9], @"dogs",
                                                                [NSNumber numberWithInteger:10], @"pigs", nil];
for (id key in [dict allKeys])
{
  NSLog(@"%@ -> %@", key, [dict objectForKey:key]);
}

I mentioned earlier than arrays are integrated into the language in Ruby. Hashes/dictionaries turn out to be really useful when they are, too. Here’s what the previous example looks like in Ruby:

{"cats" => 4, "dogs" => 9, "pigs" => 10}.each {|key,value| puts "#{key} -> #{value}"}

Hashes are easy to create; they’re referenced in the same way as an array is; the index is just any object instead of natural numbers:

hash = {"cats" => 4, "dogs" => 9, "pigs" => 10}
hash["cats"] # => 4

Having hashes be this easy to create and use lets them be used a lot more liberally. For instance, they can be used for named parameters:

render :action => 'posts', :controller => 'blog', :id => 4

This is a lot better than a selector like renderAction:forController:withID: because I can specify parameters in any order, leave out those that are optional, or modify the list at runtime like in this Facebook app code:

params = {:action => 'posts', :controller => 'blog', :id => 4}
params[:host] = "apps.facebook.com" if is_canvas_page?
render params

Another cool thing to note is on the second line: you can put if or unless at the end of lines. That leads to neat idioms like:

return nil unless input >= 0

I used hashes everywhere in my neural network to store connections between layers, weights, and all that. I wouldn’t have bothered in Obj-C, though—it’s just way too much set-up work.

Meta-programming

Now, Obj-C is a lot more dynamic than C. I can introspect options objects at runtime, and by playing with things I probably shouldn’t, I can even swap methods around. But in Ruby, it’s ridiculously easy to write code that writes code.

For my summer project, I needed to enforce the terms of a license agreement—data couldn’t be cached for more than 24 hours. So all the accessors should update the data from the server if it’s been too long, then return the new data. Say one property we’re protecting is brand. That’s a trivial method to write. It looks like this:

def brand
  update_item_data if needs_update?
  return brand
end

But I’ve got a bunch of properties I want to do this for. I don’t want to write that method for all of them; that’d be dumb. The solution?

[:brand, :gender, :price, :images, :tags].each do |method_name|
  define_method(method_name) do
    update_item_data if needs_update?
    return super
  end
end

So pretty. You can do so much with this. We would never have had to wait for Apple to make accessor generators in Obj-C 2.0 because we could have just written a method to write them for us:

def attr(*attrs) # * makes Ruby collect args into an array
  attrs.each do |attr_name|
    define_method(attr_name) do
      return self.attr_name
    end
    define_method("#{attr_name}=") do |new_value|
      self.attr_name = new_value
    end
  end
end
 
attr :brand, :tags, :goldfish

Boom! We’ve got brand, brand=, tags, tags=, goldfish, and goldfish= methods with that one last line. Hot.

So… what?

That was a lot of meandering code, but there was a point to all of it. That point is not “Ruby is awesome.” It is. But whatever; lots of languages can do all the things I just said—Python, Smalltalk, and Io are just a few examples. And I’m not saying Obj-C should be any of these things: it’s compiled, fast, and low-level—it has to make some compromises. I’m just trying to say that even though we’ve got by far the best application framework (darn tootin’!), Objective-C feels much less awesome than Cocoa. Less rapid. Less well-thought-through.

It’s hard to distill everything I miss in Ruby when I code in Cocoa, but I think these would be some really excellent first steps:

1. Closures (aka blocks): I think I’ve demonstrated throughout this post why they’re awesome. They make coding way faster, way more expressive, and way more encapsulated. And list manipulation becomes really easy.
2. Language-level lists and hashes: There is just way, way too much typing when we use arrays and dictionaries. That’s dumb. We use lists and hashes constantly.
3. No more square brackets: Square brackets are terrible. Unless you’re using TextMate with its fancy completion, you end up having to go back to the beginning of the line to add more all the time. Or at least I do. I don’t know how many call levels I’m writing when I start a line. Do you? And then things like [self doSomething] can just become doSomething because the self. can be implied. Excellent.

Yeah, I know that each of those things is a huge change. And Obj-C 2.0 just came out. I don’t actually expect any of this to happen. I’m just pointing out the awesome outside of C so that people can appreciate rapid-development techniques in the language in addition to the frameworks.

There are bridges for Python and Ruby, and Apple’s starting to support them—this is very cool. I’ve used RubyCocoa some, and from what I can tell, RubyObjC will be an even better implementation once it’s mature.

But whenever I’m using them, I can’t shake the fact that I’m doing something alien and weird and broken: Xcode isn’t a very good Ruby editor, Interface Builder doesn’t like working with Ruby classes, and there’s no graphical debugger. Maybe someday it’ll be well-integrated. Until then, we’ve got to deal with writing a whole lot more code than is necessary.

If you want to learn more about Ruby, check out Try Ruby! for an interactive Ruby prompt complete with tutorial or visit the Ruby homepage. For more on Ruby with Cocoa, see RubyCocoa or RubyObjC—you’ll have to decide for yourself which to use.