How Ruby is beating Python in the battle for the Soul of System Administration

http://devopsanywhere.blogspot.com/2011/09/how-ruby-is-beating-python-in-battle.html

by bryanwb • 15 years ago

73 44 15 years ago

devopsanywhere.blogspot.com

piccadilly2• 15 years ago

I'm not persuaded that the arrow of causality has been drawn correctly here.

I.e., it seems rather that Ruby has gained a high profile in system administration, not due to any inherent characteristics of the language or library, but because Puppet and subsequently Chef happened to be written by people who wanted to use Ruby.

Based on TFA, this was a matter of taste. I can't, for example, see why it should particularly matter for system administration whether "‘len’ was a function instead of a method)." It doesn't. But the fact that the guy who went on to write a reasonably important tool preferred to do so in Ruby on the basis of such personal prejudices made Ruby important just insofar as the tool was important, and probably contributed to Chef being written in Ruby as well.

Most of the reasoning in this article is no better than complaining that len() is a builtin. I fail to see how Perl-golf style conciseness is inherently more "productive" (particularly when it makes it harder to understand and maintain operationally important software). I fail to see how the crushing burden of spelling out 'import re' makes regex unacceptably distant in Python. I fail to see how Ruby is inherently stress-relieving or better for people who use vi, and if you don't think there is magic in Python that is probably because you have not gotten that deeply into the language. All this is pretty spurious, I think.

And if I wanted Perl, then Perl is the best possible Perl, already familiar to tons of sysadmins; and lots of good things are happening in Perl development.

What isn't spurious is if you happen to like Ruby, even if only for stupid reasons like Luke's; or if you really want to work with a tool like Chef that requires you to write Ruby. Those are perfectly good reasons for using Ruby.

But multiple languages will be used into the far future. In reality, the reason that Ruby and Python (and for that matter Perl) are so frequently put head-to-head is because they are so very similar in their abilities. That's okay. Write what you like.

bryanwb4• 15 years ago

see my summary at the end. The real cause of ruby's rise is that puppet was written in it and that you use ruby-based dsl to configure puppet. This factor is more significant than language differences.

bryanwb4• 15 years ago

you also seem to have missed the multiple times I state my personal preference for Python but am shifting to ruby for practical reasons.

Goladus2• 15 years ago

I fail to see how Perl-golf style conciseness is inherently more "productive" (particularly when it makes it harder to understand and maintain operationally important software).

I'm not sure the author made a great case, but it does matter. Unix admins often think in terms of lines, because lines are the default unit of action. No matter how much planning and cfengining you do, there are still times when an admin must take direct action on a system. When that happens, they use the interactive command lines. These lines often aren't repeated frequently and have no need to be stored in any file other than .bash_history, and therefore do not need to be read or maintained. In those situations, the conceptual and physical overhead incurred by something like 'import re' is most assuredly significant.

piccadilly2• 15 years ago

I can get insanely concise one-liners out of C and shell, but that doesn't make those inherently more-productive languages.

I forget what bash does, but when I write a block of shell in zsh, it is saved as a 'line.' This is a tool issue, not an issue of language expressiveness.

If you can't write concise Python, maybe that's just because you aren't that comfortable with Python... that's a perfectly good reason not to use it but it isn't some sort of dramatic limiting case requiring all system administrators to use Ruby instead.

kamaal2• 15 years ago

I fail to see how Perl-golf style conciseness is inherently more "productive" (particularly when it makes it harder to understand and maintain operationally important software).

Let me ask a simple question here, how does one possibly remember all the vi commands or emacs commands for that matter? The answer is no one sits down with sheet of paper and memorizes these sort of things. A person sits down and understands how to use a particular tool, and then later on he just looks up to some form of a reference. Over time this becomes just muscle memory.

This where tools like Perl/Awk/Sed and other Unix text processing tools win over Python. You have to spend some learning how to use them initially after which you get very productive with them. For most people who don't have exposure to how much one can push the combination of pipes and Unix text processing utilities its a little difficult to explain it in words. You have to just use it to feel how power full they are.

Often you can save days of effort writing lengthy programs and testing them. All you need to be is familiar with the Unix text processing utilities and know how sew them using pipes and you can amazing lot of work without writing any code at all.

Perl has a special distinction that it not only helps you do all this things seamlessly but also makes a great scripting language for any task imaginable today. You get one platform for quick scripts plus application development of almost any kind.

They didn't call Perl the swiss army knife and duct tape of the internet just like that. I don't think any other programming language has come to adapting practical software realities as much as Perl has.

Every other programming language forces you think in one way or tries to force its paradigm you. On the other hand, Perl bends towards your paradigm. That to me is more than sufficient reason to use and go back to Perl again, because adaptability ensures survival on the longer run.

A painter is never happy when the brush dictates his art. A painter is happy when the brush paints the way he wishes to dictate his art.

eropple2• 15 years ago

Until someone else has to maintain your code. I've noticed that most folks who default to Perl tend to write Perl in a manner largely inconsistent with the next guy's. "Bending toward your paradigm" is not an inherent good if anyone else ever has to deal with your paradigm after you leave.

Consistency at 90% your-arbitrary-metric-of-quality is generally better than wild inconsistency at 100% your-arbitrary-metric-of-quality, at least when multiple people are around.

rytis2• 15 years ago

Neither Python nor Ruby are best suited for one-liners that the author is emphasising. One liners are bash-fu or commandline-fu. Anything above 100 lines (just a figure from the article) is what Python or Ruby should be used for.

Last 10 lines in a file? 'tail -10 filename.txt'

Right tools for the right job.

As to Python vs Ruby? I believe the argument isn't about the language (although I think Python is more 'sane' :) ), but the community that surrounds it. And I like the Python community.

bad_user• 15 years ago

     Last 10 lines in a file? 'tail -10 filename.txt'

A lot uglier, but you can do a one liner ...

     ruby -e 'lines=[]; while gets(); lines << $_; \
       lines = lines[-10,10] unless $. < 10; end; \
       puts lines' < filename.txt

The problem with using "the right tool for the right job" is that there are many tools and many jobs, and a general purpose language helps you get things done in case you're stuck. In general I use Ruby for doing string manipulation before piping to other commands.

Here's an (ugly) one liner that shows the biggest tables in a MySQL database:

     mysql -u root godzilla -s -e 'show tables' \
     | ruby -ne '$_.strip!; puts %Q{SELECT count(*) \
       as cnt, "#$_" as tbl FROM #$_; }' \
     | mysql -u root godzilla -s \
     | sort -nr \
     | ruby -ne 'parts = $_.split /\s+/; \
       puts "%40s : %s" % parts.reverse' \
     | head -10

You can probably come up with something more clever, but I barely gave that one any thought.

jbert• 15 years ago

> A lot uglier, but you can do a one liner ...

Yeah, but that one-liner reads the whole file. 'tail' can be more clever (and is). 'strace' output:

    open("a", O_RDONLY)                     = 3
    fstat(3, {st_mode=S_IFREG|0644, st_size=545975512, ...}) = 0
    lseek(3, 0, SEEK_CUR)                   = 0
    lseek(3, 0, SEEK_END)                   = 545975512
    lseek(3, 545972224, SEEK_SET)           = 545972224
    read(3, "tu 11.04 \\n \\l\n\nUbuntu 11.04 \\n "...,     3288) = 3288

'tail' is seeking to the end, then reading backwards.

The point being, these 'little tools' can contain important optimisations - they aren't necessarily equivalent to a naive reimplementation.

dramaticus3• 15 years ago

tac filename.txt | sed 10q | tac

nabb• 15 years ago

tail can be done in ruby can be quite succinctly with the $< (ARGF) variable:

  ruby -e'$><<$<.map.last(10)'

For the rest of the ruby one-liners in the page the author references, most can be done more easily with standard command line tools (although most people aren't well-versed in sed, so 'ruby -pe puts' might be better than 'sed G').

viraptor• 15 years ago

There's one big problem with code like that. If you're doing devops-y stuff, then you really, really don't want to debug something like that when your pager goes off at 3am. The cleaner the solution the better, because after you wake up `$><<$<` is just a blurred thing with no meaning...

davnola2• 15 years ago

Cool. That hurts my eyes.

    ruby -e "print ARGF.readlines.last(10)"

prolepunk• 15 years ago

Not that you can't do this in python:

python -c "print '\n'.join(open('LICENSE.txt','r').read().split('\n')[:-10])"

But this is besides the point.

jodrellblank• 15 years ago

Your split and join seem redundnant, and open defaults to read-only.

  print open('whatever.file').readlines()[:-10]

But still, don't do that on any log too big to read into memory.

Loic• 15 years ago

This article should have the following title: "How Ruby is Beating Python in the Battle for my Soul as System Administration". The summary shows that the person has not been working in a team to manage many components:

> Ruby's greatest strength is its amazing flexibility. There is a lot of "magic" in ruby and sometimes it is dark magic. Python intentionally has minimal magic. It's greatest strengths are the best practices it enforces across its community. These practices make Python very readable across different projects; they ensure high quality documentation; they make the standard library kick ass. But the fact is that we sysadmins need flexibility more than we need raw power or consistency.

As a sysadmin you insanely need consistency all over the place or you cannot scale. This is in fact why Fabric, Puppet and Chef were created in the first place, to have consistent and automatic ways to deploy on and maintain your systems.

llambda• 15 years ago

> As a sysadmin you insanely need consistency all over the place or you cannot scale.

This is exactly what ran through my had reading the bit you quoted above: consistency is king! Try scaling magic. Maybe some people are able to do this. I, personally, am a mere mortal, and that means consistent, clear code is a very powerful tool to me. More so than this "flexibility" the article's author is citing. In fact I would argue that consistency and clarity lend flexibility if you understand what you're doing.

brudgers• 15 years ago

Apparently there is an unstated premise -- sysadmins using Powershell lack souls.

antoncohen• 15 years ago

The short/one-liner examples are more likely to be done from command-line shell, where they would be much easier:

  head -10 /path/to/file

  dmidecode | grep -iq vmware && echo "is vmware" || echo "is not vmware"

The list of projects written in each language has more to do with what else you use than the languages, e.g., if you are working with Ruby web apps you may use Rake and Capistrano. SCons, Mercurial, Bazaar, and YUM are all written in Python.

glenjamin• 15 years ago

This essentially reduces to "it's easier to write a fluent, readable, flexible and powerful DSL in Ruby than it is in Python.

Sysadmin tools benefit greatly from using such a DSL as their interface vs custom parsers or configuration files.

Hence, Ruby is a good choice for these sorts of things.

IgorPartola• 15 years ago

I use puppet in a mostly Python shop. I like that it uses a DSL and not raw Ruby/Pyhton/etc. However, I diagree with TFA about Ruby becoming the dominant language for a very simple reason: all UNIX-like systems have sh and in recent ones it supports all the same things. Most distros which I would consider installing onto servers, managed workstations, etc. have a Python interpreter and a standard library which includes everything from process management to a web server. These two tools are ubiqutous. However, not many distros ship with Ruby installed. Writing a 200 line Ruby script means that it is noe your job to ensure you have Ruby on all your machines, not the dist maintainers'.

sliverstorm• 15 years ago

This is a rather misguided article. Unless everything has completely changed, Ruby is used for almost nothing in mainstream Linux. Bash and Python dominate "stock" system management code, and Bash and Perl seem to dominate for top-level add-on scripts written by a given sysadmin.

I've never even heard of any of those Ruby projects besides puppet, and I've only ever once installed Ruby- to support some obscure v0.03a library a developer wanted to try.

(Was a part-time sysadmin for 2 years)

bryanwb4• 15 years ago

The unstated premise of this article is that all sysadmins will very soon be using puppet or chef, thus programming in some subset or DSL that looks a lot like ruby (puppet).

However, that's a separate article that I need to get around to writing.

kstrauser• 15 years ago

The Python example is completely broken. It'd be something like:

    import os
    if 'vmware' in os.popen('dmidecode').upper():
        print 'this is a vmware vm'
    else:
        print 'this is not a vmware vm'

I like regexps as much as the next ex-Perl hacker, but sometimes a little string manipulation is a lot better.

bryanwb4• 15 years ago

fixed, tks

subleq• 15 years ago

The condition will always evaluate to False, a lower-case string can't be in a string that has been .uppper()ed. popen(...).lower() will work.

cowmix• 15 years ago

Again, another article that has a lot of talk about CM systems (CFEngine, puppet, chef, etc) and no love for bcfg2.. which happens to be Python based.

fduran• 15 years ago

"Puppet and Chef are seeing rapid adoption" != "ruby is inexorably becoming the dominant scripting language for linux system administration."

rbanffy• 15 years ago

After seeing backticks, %x and $? being used, I have to say Ruby, when employed like this, competes more with Perl than with Python.

adient• 15 years ago

As a sysadmin who uses Puppet for a few hundred servers, I strongly disagree with the assertion that "After spending 25% of your time working with Puppet, you will be much more likely to reach for ruby for your next scripting task." I'm not even sure how you reach this conclusion since Puppet uses a DSL and not Ruby. I've been using Puppet for a while now and I don't know and have never used Ruby, and I'm not planning to. Much of what you have written seems like you're really stretching to validate your decision to start using Ruby instead of Python, which I'm not even sure why it matters. Use what you/your organization prefers, otherwise it doesn't really matter.

mitchty• 15 years ago

I somewhat agree even as a Ruby fan of 14 years.

We have puppet now at work, and going through the recipes, its rather obvious at times that people can use Ruby, without really learning the language.

I managed to shorten a number of the recipes and configs from 60 lines of... well WTF basically, to 10-20ish lines of much more readable ruby.

I will however agree with the assertion that if the management tools that are most popular are written in Ruby, one would be more likely to do more within them.

wildmXranat• 15 years ago

There's one thing left to do: Write a better tool than chef or puppet and do it in Python. That or you know - use the correct tool for the job and be done with it.

zzzeek• 15 years ago

tl;dr; ruby is the new perl

vegai• 15 years ago

Only a moron would refuse to use a good tool if it's written in a language that is not said moron's personal idol.

Except of course if the language is Java.

aer0• 15 years ago

Ruby and Python break backward compatibility too often even with minor language version upgrade. It's harder to maintain working language version across servers than do real sysadmin tasks.

Perl rocks and robust to that.

See Perl equvalent to Ruby's Capistrano/Puppet/Chef.. or Python's Fabric/func..

Rex - http://rexify.org/

How Ruby is beating Python in the battle for the Soul of System Administration

Related Stories