Responsible Monkeypatching in Ruby

https://blog.appsignal.com/2021/08/24/responsible-monkeypatching-in-ruby.html

by thunderbong • 5 years ago

55 55 5 years ago

blog.appsignal.com

develop7• 5 years ago

I wonder why this otherwise thorough post doesn't mention almost decade-old feature called "refinements" [1]?

1: https://ruby-doc.org/core-2.5.3/doc/syntax/refinements_rdoc....

eloisius6• 5 years ago

Not sure why the author chose not to, but personally I’ve avoided use of refinements because of the weird, stateful way that it changes the implementation of a class. It’s too confusing to scan the code quickly and know whether a particular class has a refinement activated in a particular scope. Ideally don’t monkey patch things, wrap them in your own class that decorates a stdlib class. If you can’t do that, my personal preference is to have the monkey patched class be consistently monkey patched on all scopes.

Lio2• 5 years ago

I don't know the authors reason but I've personally always avoided refinements after first reading about their performance hit for JRuby code[0].

I read somewhere recently that even for MRI ruby you can take a 40% hit when using refinements (but please take that with a big pinch of salt as I can't find the source right now).

0. http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/...

byroot• 5 years ago

The source is me: https://gist.github.com/casperisfine/1c46f05cccfa945cd156f44...

Note that this overhead apply as soon as a refinement is defined for a method regardless of wether it's ever active.

That 40% figure is for an empty method. So for "big" methods that are infrequently called it's probably fine, but it should really be avoided in hotspots.

petepete• 5 years ago

I use refinements in a couple of my projects, they definitely feel like the right approach, especially in a library where you don't want unintended consequences.

Admittedly they're a bit of a mystery to lots of devs.

pankajdoharey• 5 years ago

Absolutely, refinements are the most legit way of monkeypatching. I could almost bet the author doesnt know about it, had he known there is no way anyone would forego mentioning it when it was specifically introduced to address the title of the post.

nerdponx• 5 years ago

Can someone help me understand why anyone would ever rely on monkeypatching in non-test code?

Python is just as dynamic as Ruby in a lot of ways. But in Python, overwriting an existing and important/fundamental/ubiquitous method would be an absolute last resort when all other sensible approaches have failed. And in that case you would at least have the sense to save the original method somewhere under a different name.

What makes such a horrifying idea so apparently not-horrifying in Ruby? The ability to "reopen" class?

None of the examples in the blog post seemed like such dire circumstances as to require it. Maybe replacing a buggy method while you wait for a PR and new release is a valid use case. But the failure mode is also particularly nasty.

joelbluminator10• 5 years ago

The best example I can think of is Rails' ActiveSupport. The changes it brought to Ruby aren't fundamental in any way but they are convenient. For example asking whether a variable is present? (truthy, so empty str and empty [] should return false) is not possible with plain Ruby but is possible in Rails. It's stuff web developers need and makes things simpler. This is done by monkeypatching. Another example is if you're working on a library that has some bug but you can't quite yet jump to the next version.

eloisius6• 5 years ago

My favorite ActiveSupport monkey patching is on number classes. Things like 2.weeks.from_now are just too useful to scorn monkey patching on some programming ideological reasons.

lovegoblin2• 5 years ago

or example asking whether a variable is present? (truthy, so empty str and empty [] should return false)

Actually this is kind of the whole point of #present? - it's not just a truthiness check.

The only falsey values in Ruby are `false` and `nil`, but in the context of a Rails app you might want to treat things like "" and [] as "not present" (e.g., a user leaves a text field blank).

orhmeh09• 5 years ago

I think it may be a cultural thing — I don’t have the Smalltalk blue book with me, but I think this type of modification at runtime is very common in Smalltalk environments and languages related to it.

billyruffian• 5 years ago

I'm in the middle of writing some of this right now.

For 'reasons', I need to add callback hooks into a particular HTTP Client library my org uses across several dozen of projects. The library we commonly use doesn't have hooks.

I've got a couple of options: switch wholesale to a different library, build a child class, build a nasty bit of monkey patching boiler plate, or build a little library which prepends some logic into the ancestors chain.

The first and second would cause eye-rolls ("I'm just calling an internal api in the way I've always done, why are you making me change it everywhere!"). The monkey patching boilerplate, well, gross. But a little bit of prepend logic packaged into a gem and I've got the behaviour I need.

One of the tremendous joys I get from ruby is never having the feeling that the language or runtime can't do something I need. Whether that thing should be done or which way it should be done is another matter.

Fire-Dragon-DoL11• 5 years ago

As you can see from below, it's cultural. It's also the reason why every Rails codebase turns really bad as it grows: metaprogramming keeps being used extensively instead of being limited. Modules abused, stuff monkey patched, methods creating other methods continuously.

The consequences are always: code unmaintainable over the long term.

Unfortunately to reach this disillusioned state you need to hit some really hard walls, which doesn't happen to all ruby devs.

rancar2• 5 years ago

At one startup in 2007, I built a Yodlee style automation framework on top of Ruby and the web automation gem Watir. It ran across most of the big banks in the US, whose servers failed frequently with one off errors that would be resolved with a refresh. As I built my framework atop of the Watir gem, the easiest way to global fix this issue (and others) was to add error catching and retry methods upon server errors using this monkey patching technique. I wrote this as its own class and loaded it as a sidecar to the Watir gem. This way our custom implementation would enhance Watir across all our deployments while allowing us to continue upgrading the Watir gem in the future. With this specific enhancement of retry upon server errors, we were able to have robust error handling for our application globally that was specific to our use case without having to add template code error in each bot written to crawl each of the US banks which would have been required for each page load and click that required a server response to proceed. Each bank bot would take a few hours to write and was reliable.

Cullinet• 5 years ago

I'm entirely unfamiliar with Ruby so although this may seem like a swipe at your language choice it isn't, I am confused about the banks not wanting to use CICS and extend to the web using something like Liberty server profiles that launch java threads that understand CICS and therefore any custom error codes specific to the transaction system. I'm guessing that you needed to catch errors that weren't native to your code and unique to the respective bank clients. I'm further guessing that because CICS provides a comprehensive custom error handling protocol and services, and CICS is everywhere in financial transactions and nobody wants to be responsible for the fate of any lost ts error messages, this is why you could patch to handle the errors in this way without any problems with your code?

Since IBM went all services and consulting and practically only mainframe remained of the old IBM, and whilst x64 Etal went into architectural Cambrian Explosion, at the turn of the century, IBM Z division has become a entirely different animal. The just discussed at Hot Chips 33 Telum processor with CISC instructions for memory architecture networking and in pipeline RNNs is so out there that I have started to try and figure out how much it takes to get to be a IBM partner just to get access enough to write about this technology freak. If the team exists the finance is slam dunk link with a infrastructure fund. This mainframe AI can be trained with regular tools and functions with really low latency - it's on the die. Just don't be ageist and train vets with clearances (for LOB or just because it's the right thing to do - my partner founded a headhunter firm for veterans and placed the vast majority in IBs after 6 months of his programme - and age no issue to CEO signed undertakings, my partner really hit it out the park with his programme quality, quant covered from fp math to martingales but it had to shut down because the people guy who kept everything from rabbit holing to fulfil my partners fantasy hedge fund ambitions died of cancer and a tech partner is necessary to free me to pick up the relationships which are as open as the day my partner kicked those doors off their hinges with some outrageous leverage of 80s mainframe knowledge that pumped the flow desk of the financial infrastructure organisation who's CEO didn't blink when my partner made the FT trampling over every unwritten rule of headhunting that he never knew existed like everything else beyond his screens. This guy has just shy of 50 years success behaving like this what am I to know? I am still recovering from the unending cardiac arrests I had for no reason until I realised that some relationships really do transcend everything. The UK incidentally has a enormous pool of senior management talent created by the government firing all civil servants who qualified vocationally instead of university. Everybody was summarily dismissed. Just before the 1997 employment act was introduced. I have worked with the director of the largest aide agency in budgetary terms after the Gates foundation whose office was a dozen or so people. It now needs 600 apparently. The logic was partner agencies didn't manage fraud well enough for the British government liking. Plenty better than we do without any relationships with the same leaky institutions that now take pride in embarrassing the Brits with implicit sentimental support from the agencies we cut off.

My point is not a self promotion but the promotion of a architecture hardly anyone gets near enough to speak about for comparison with the front line of web development. This is a unmitigated loss for everyone all around.

All that boastfull background is intended to directly criticise the nature of this technology discussed here that's controversial enough to elicit numerous warnings in the comments immediately after publication.

Given the search rank of HN and the difficulty with finding human readable (non expert) commentary on highly technical issues like monkeypatching I don't see how the existence of this discussion doesn't de facto eliminate the techniques and even potentially the technology as a serious option for critical systems deployment.

Consider the weight a browsing executive will place on what's said here.

In the example just given the application is applying node specific handling changes to in progress production code in a webserver talking to banks running transaction systems.

Consistency of the www service and presumably the user interface critical to customers and likely to be the only way to access account information (giving up our company x.500 terminal probably not even after corporate death because companies have been bought from administration only for grandfathered services often enough I considered trading them for a living) despite how much people will pay for the capability.

Monkeypatching tells me that the errors aren't in your code because why introduce the trade-offs and risks of the technique for blowing up all of your most impressive customer references at the same time? I'm guessing hard nothing much has to be done to the error messages.

But it's so hard to blow up CICS (assuming you read the docs) potentially even lacking in-depth mainframe knowledge the opportunity to launch lightweight java environments and FFI to your Ruby runtime or make the call via CICS from a SPI error manager in javascript that because CICS you know is giving you actual state, or creating a message feed for individual subscription to have a extra confirmation that you didn't lose their messages, and running this under the banks sysadmins purview whilst giving you the same certainty your updates applied with point in time rollback to exact system clock and transaction stamp and in flight transaction state and zero sweat about any potential consequences looks pretty attractive to where I am seeing this.

If I have totally misunderstood this I'd like to beg a point for the scenario that I have described : namely one where you are responsible for the ephemeral bleeding edge of web UX and (in my hackneyed eyes) making a stateless best effort ad hoc text delivery protocol statefull reliable and infinitely scalable also a GUI thread manager and connections manager and anything else I omitted to boot, behind which is a great hulking mainframe nobody wants anyone to touch because the lack of adoption can't overcome human fear of being responsible for borking a IBM Z no matter how indoctrinated everyone is that Z can't fault assumptions of complacency obliterate reason and so you're hoodwinked (joined that club long ago) into not looking up how you can run webserver tech on CICS keeping it on the mainframe and getting the benefit of all that reliability and fault tolerance and data processing invincibility for your own personal gain.

Putting the code on CICS is slam dunk argument because how can anyone beat the mainframe management magnificence mantra unless there's some major issues with the big iron priesthood. Your code automatically gets the same management and reliability capabilities and gives sysops single pane of glass sight of everything happening with the front end interactions opportunity to get in on the new wave potentially for them personally and a nice brag of relevance in the face of the bleeding edge and unstructured world we represent, meanwhile the web development team just bagged major C Suite kudos and the ability to coopt some of that incredibly valuable institutional invincibility inventive that can transform your business model from job quotation to Senior Web Scale Z/OS Customer Systems Integrity Consultant times head count and paid pitch fees, retainer and expenses. Plus probably the possibility of getting insurance cover for DR scenarios and hourly overtime on any incidents paid for by customer DR budgets and policies instead of being the wrong side of pointing fingers and the one thing that C Suite types sympathise with is web UX and general web reliability grief.

I surely deserve criticism about my style with this, but what's going on with the mainframe about now is really wild. My pitches above were hypothetical to illustrate the point although my partner and good buddy is as much work as that non fiction description and a leading dinosaur in my eyes. I absolutely accept that the chances of mainframe ubiquity are not meaningful numbers. But with everyone busy making the OS as irrelevant as possible, how long does it take for the mainframe Telum cpu to be viable mainstream silicon? Very strangely I think I am going to live to see that move being made. Maybe the IBM strategy is the right one and only the attitude of management the problem.

thrower123• 5 years ago

Depending on the strain of web development you encounter, it's more or less common in Javascript. Part of that might be overlap with Rails people, but there's such a long history of having to patch broken and missing JS language features and standard library functions in incompatible browsers that monkey-patching is normalized under the name of polyfills.

Once you're used to doing that kind of thing, people will feel the urge to take things to the next level, augmenting standard functionality and replacing the original functionality globally. Maybe you replace console.log() with a wrapper that includes a timestamp in the output, for instance. Not insane, and manageable for a smaller project that you have your head around.

It falls apart awfully quickly as it scales though. If behavior in your system diverges from standard because of a maze of monkeypatching, then you have to learn those quirks or there is a lot of frustration. And the possibility of spooky breaking changes in seemingly unrelated components becomes a big problem.

hamburglar• 5 years ago

While this is technically the same thing as the rampant monkeypatching that happens in ruby, there is a difference in practice, in that polyfills tend to be written to emulate well defined interfaces that simply aren’t there, whereas monkeypatching adds random magic like the aforementioned “weeks” method on numbers.

This is personal preference, but I find it utterly absurd to adorn numbers with these time dimensioning methods. Why not volume or length too? Should I be able to say 2.miles? Why would an integer be innately bound to time, as opposed to a unitless mathematical abstraction? Ruby folks like to add these gimmicks that make code “read like English” and I find it very cutesy, for lack of a better word. When I see stuff like this, I’m distracted by pondering what the hell the magic actually means. Can I say 6.from_now? Yes, actually, I can. Does it make any sense that what I call “from_now” on is technically a unitless number? No, not at all. It is a pretense that it’s a dimensioned number for the sake of cuteness.

ysavir• 5 years ago

There is no such thing as "responsible monkeypatching". Even if it's fixing a bug in the source code or annotating, as the author suggests are practical uses.

Writing good code is about clarity and working within maintained expectations. People reading your code should be able to know what it does. If someone joins your team and knows that Method X in library Y is bugged, they will expect it to be bugged in your codebase as well, and reasonably so. The fact that you monkeypatched it is not something that they will be aware of, and can potentially cause bugs in their code as a consequence, costing them time and patience.

A better approach is to use a wrapper instead of monkeypatching, or inherit from the bugged class and overwrite the bugged method. You get the same results, but anyone reading your code, with or without prior knowledge of how your code works, will:

1. Have immediate understanding that you've altered the source code in some way

2. Have a crumb trail leading them back to your custom implementation

BiteCode_dev• 5 years ago

Exactly, when the article used their example of bug found in production, I knew what would happen before the reveal, because it's what always happen.

Damn it, you got a language with namespaces, why do you think that is ?

Responsible Monkeypatching in Ruby

Related Stories