Monkey Patching Reprise

It turns out that things aren't as simple as they seemed in my last post. A couple of people have pointed out to me that Python can indeed do monkey patching - so what's the difference between this and Ruby? Am I just making a fuss over nothing? First of all, to some proper definitions.

What Even Is Monkey Patching?

It's also known as guerilla (/gorilla) patching, hot-fixing, and more recently, 'duck-punching'.

Geoffrey: Now, you went to PyCon a couple months ago. And it’s well-known that in the Python world, they frown on monkey-patching. Do you think they would think more positively of duck-punching?

Adam: No, I think they will continue to look down on us, no matter how awesome and hilarious we become.

Voice In Background: Isn’t that the truth.

Geoffrey: I also have Patrick Ewing. Is this a good idea, and will it catch on?

Patrick Ewing: Well, I was just totally sold by Adam, the idea being that if it walks like a duck and talks like a duck, it’s a duck, right? So if this duck is not giving you the noise that you want, you’ve got to just punch that duck until it returns what you expect.

Transcript from Geoffrey Grosenbach podcast at RailsConf2007

Monkey patching can mean some subtly different things:

• Changing a class's methods at runtime
• Changing a class's methods at runtime and making all the instances of that class change after the fact

As pointed out in this thread on StackOverFlow by Dan Lenski, both variants are indeed possible with Python. Here's an example:

class Widget:
    def __init__(self):
       pass
    def who_am_i(self):
       print("I'm a widget")

>>> my_widget = Widget()
>>> my_widget
<Widget object at 0x7f6b5aa52e80>
>>> my_widget.who_am_i()
I'm a widget
>>> def teapot(self):
...     print("I'm a little teapot")
...
>>> Widget.who_am_i = teapot
>>> my_widget.who_am_i()
I'm a little teapot
>>> new_widget = Widget()
>>> new_widget.who_am_i()
I'm a little teapot

And Python doesn't warn you either! So perhaps I was unfairly harsh to Ruby? Not quite.

One Important Difference

Unlike with Ruby, we can't monkeypatch the basic built-in classes, such as int, float or str. This is because they are defined in C extension modules which are immutable. These modules are shared between multiple interpreters and made immutable for efficiency (and safety)'s sake. So when you try to change the behaviour of the built-ins, you can't -

def own_up(self, a_string):
    return a_string.length()*"!"

>>>my_string = "Hello, World!"
>>> my_string.upper()
'HELLO, WORLD!'
>>> str.upper = own_up
Traceback (most recent call last):
  File "<input>", line 1, in <module>
TypeError: can't set attributes of built-in/extension type 'str'

To do something which approximates the effects of monkeypatching a built-in we have to subclass, like so - this example lets us write a custom instance of the str.upper() method by inheriting from str.

class CustomString(str):
    def upper(self):
        desired_value = "!" * len(self)
        return CustomString(desired_value)

>>> custom = CustomString("hello world")
>>> custom
'hello world'
>>> custom.upper()
'!!!!!!!!!!!'

Ta-da! I still maintain that this is saner behaviour than Ruby.

(Sidenote - Remember, for str and any class based on it, Python strings are immutable - when we call methods on a string, we just get a new return value that we have to assign to some other string object to store.)

If Python and Ruby do it in similar ways, is Monkey Patching even All That Bad, then?

Yes! I still maintain that monkey patching is dangerous and shouldn't be used often. I think it says something positive about Python that it's not something you need to know intimately in order to program competently in the language, and indeed most Python programmers I know feel a bit iffy about it. It's a little off-putting (to say the least) to realise how much of a way of life it is for Ruby programmers.

But don't just take my word for it - have a read of this insightful blog post by Avdi Grimm, an experienced Rubyist worried about the impact of thoughtless 'hip' monkey patching - a choice quote:

Where I work, we are already seeing subtle, difficult-to-debug problems crop up as the result of monkey patching in plugins. Patches interact in unpredictable, combinatoric ways. And by their nature, bugs caused by monkey patches are more difficult to track down than those introduced by more traditional classes and methods. As just one example: on one project, it was a known caveat that we could not rely on class inheritable attributes as provided by ActiveSupport. No one knew why.

A fun (scary) read.

Temporary Monkey Patching in Python for testing

As I was looking around at this stuff, I discovered there are a couple of libraries, unittest.mock.patch and pytest monkeypatch designed to allow temporary monkeypatching for testing - when the function or statement using it exits, the patch disappears. I can see how useful it would be to set up mocks in this controlled way, rather than having to worry about it affecting all of your tests. It was deemed so handy that mock is now part of the standard library in Python3. Time to investigate further!

First Steps Exploring Ruby

Last night I went to the first of the Cambridge Programmers' Study Group new sessions - we're starting to work through the book 'Understanding Computation: From Simple Machines to Impossible Programs' by Tom Stuart. The book uses practical exercises in Ruby to explain a bunch of theoretical computer science stuff, working up to cellular automata, Turing machines and the lambda calculus (exciting!). As Stuart's blurb explains, the book is aimed at

An audience of working programmers without assuming any academic background.

So, people like me. It sounded like exactly the sort of thing I should be learning, since I don't have a Computer Science background.

One small problem - very few of us in the group actually knew any Ruby! So we spent this session trying to teach ourselves using the examples in the first chapter. It was fun trying to pick up a new language in the company of other people - a lot more companionable than noodling around on your own, and easier to find out cute things about the language because if one person discovered something cool, they shared it with the group. This post will be about some of my first impressions of Ruby.

Getting Ruby set up

For those like myself who prefer more instructions than 'download Ruby online!' here's how I did it in the end: ` $ sudo apt-get install ruby-full which on my Ubuntu system downloaded Ruby 2.3, which was just fine for my needs. If you use Windows or Mac I'm told the instructions on the Ruby-lang website are quite helpful and up-to-date. I first tried using the default terminal, irb, which you call from the command line

user@device:~$ irb
irb(main):001:0> 1+2
=> 3

This works fine for the first few exercises, but I couldn't get auto-indentation to work, important for writing multi-line programs! I tried tinkering around with adding a .irbrc file with require 'irb/completion' as recommended in the comments of the main irb.rb file (which I found in /usr/lib/ruby/2.3.0). This didn't help - perhaps I put the config file in the wrong place? I ended up just using a slightly different terminal, pry, which was recommended by a colleague at the Meetup. It does auto-indentation and colour-coding too, which was very helpful! I don't think I'll be going back to irb. To install that, I just did $ gem install pry (gem is Ruby's package manager). To start using it, just type pry in the command line.

Ruby is like Python

Ruby is similar to Python in a lot of ways - this almost hindered me as I kept getting confused between Python and Ruby syntax. It felt like trying to learn Spanish when I already knew some French - confusingly close but distinct. I think ultimately this is helpful (I was mentally translating some things to Python to understand them, and that worked quite well) but to start with it was rather confusing.

Some similarities: yield sort of works the same

>> def do_thrice
*   yield
*   yield
*   yield
* end
=> :do_thrice
>> do_thrice { puts "Hello world!" }
Hello world!
Hello world!
Hello world!
=> nil

And the string formatting works rather like Python 3.6's new f-strings. So for instance, in Ruby "#{x*2} and also #{y+1}" gives you a string that calculates & inserts those values.

The * operator works in similar ways to unpack elements of an iterable - for instance, you can do fun things like:

>> a, *b, c = [1, 2, 3, 4]
=> [1, 2, 3, 4]
>> b
=> [2, 3]

Ruby is not like Python

Objects! Objects everywhere!

(Almost) everything is an object in Ruby, and it made me realise how I usually write Python in a fairly un-Object Oriented style. I generally avoid writing classes if I can help it (especially classes which only contain two methods, one of which is init but this isn't really possible in Ruby. I suppose it's good for my soul - having a language that forces you into object-oriented thinking must make you confront what that really means. That's why there are these little => nil statements scattered around my code examples, by the way - irb and pry always shows you what object your statements evaluate to, and often that's nil, which is the object representing nothing - the 'empty' object, if you're mathematically minded.

I guess I'm going to have to go read those books on Design Patterns after all...

It's super-terse

If Python cares a lot about preventing programmers having to type unecessary characters (goodbye ; I didn't miss you) then Ruby cares even more. There are a lot of things you can do in one line. Here's a good one:

>> (1..10).select { |num| num.even? }
=> [2, 4, 6, 8, 10]

This creates a range which spans the numbers 1 to 10, then looks at all the numbers in that range and, if they are even, adds them to a list and returns the list. Phew!

At times, this can be confusing - for instance, there's no need to write explicit return statements if you don't want to. By default, Ruby returns the evaluation of the last statement in the body of a method. For instance,

>> def what_do_i_return
*   "A"
*   "B"
*   "or C?"
* end  
=> :what_do_i_return
>> what_do_i_return
=> "or C?"

You can write explicit return statements, but you don't have to. And the way to get a function to evaluate to sort-of-nothing is to explicitly return nil (of course!). This returning-by-default is confusing at first, but it seems to fit with the rest of Ruby's style.

'nil' isn't really nothing

You can put nil into things, which is hella confusing. For instance, suppose you have an array, you can add nil elements to it:

>> my_array = ["a", "b", "c"]
=> ["a", "b", "c"]
>> my_array[4]
=> nil
>> my_array.push("d")
=> ["a", "b", "c", "d"]
>> my_array.push(nil)
=> ["a", "b", "c", "d", nil]
>> my_array[10] = "surprising behaviour"
=> "surprising behaviour"
>> my_array
=> ["a", "b", "c", "d", nil, nil, nil, nil, nil, nil, "surprising behaviour"]

Assigning to an element beyond the length of the array inserts nils until it reaches that element index (which didn't exist before!) and then it bungs in the thing you assigned. Intriguing!

Some gripes

'Monkey patching'

Monkey patching is dangerous. It's the way Ruby programmers refer to the ability to dynamically modify classes 'live' - at any time. You can even do this with Ruby's built-in classes, like String. At first glance it sounds really cool - you can just go define a new class method whenever you like! For instance

>>class String
*    def shouty
*        upcase + "!!!"
*    end
* end

adds a new 'shouty' method to strings. So now any string can use .shouty, just as if it were a built-in method

>> "hello".shouty
=> "HELLO!!!"

But here's the thing - monkey patching can break things really easily - and Ruby won't warn you about it! One of the guys sitting next to me at the meet-up managed to break his irb's Array class by redefining some of its built-in methods. Especially if you're new to Ruby and don't know all the built-in class names, it's really easy to accidentally monkey patch your way into disaster. It seems like a really powerful feature, but I'm not happy with how it seems to work. Perhaps with a nice Ruby IDE that reminds you about all the classes you have on-the-go, it's easier to avoid horrible mistakes?

CONSTANTS - why even bother?

'Constants' are sort-of a thing in Ruby. If you define a variable with CAPITAL LETTERS, the Ruby interpreter will give you a warning. But it won't do anything about the warning - it'll still change the 'constant' value, and just complain about it. Here's an example:

>> CONST = "Don't change me please"
=> "Don't change me please"
>> CONST = "I changed you anyways!!"
(pry):224: warning: already initialized constant CONST
(pry):223: warning: previous definition of CONST was here
=> "I changed you anyways!!"

What's the point of having this feature? Can we perhaps write larger programs that will catch this warning and not change the constant (well, and change it back, since we warn but change things anyway)? Basically not at all, apparently! I'm happy to be enlightened by some Ruby-god but as far as I can tell, this feature does nothing except confuse people used to programming in languages where constants exist and are, well, constant.

Overall Impressions

I found that during the session I fluctuated between delight ("what a nice way to do that!") and terror/annoyance as things behaved in unexpected ways. It has some peculiarities that particularly stand out to me in comparison with Python, and some features that are just adorably cute. I've been noodling around with it on and off all day today, and still enjoying getting to know it. Overall I'd say it's been fun! I can't say that I'm dying to use it as my new scripting language, but I'll certainly be pleased to become better acquainted as we work our way through the Computation book.

PS

Thanks again to RedGate for hosting the session and feeding pizza to 20+ hungry programmers!