GNU grep 2.12 changes behavior of recursion options, breaks existing scripts

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=678652

by cschramm • 14 years ago

119 77 14 years ago

bugs.debian.org

buster2• 14 years ago

Who thought it might be a good idea to break -r (most likely the most often used option) and use the old behaviour for -R?

Wouldn't it be better to atleast let -r behave like ever and change -R?

Anyone ever used -R here? :)

leif2• 14 years ago

Yeah, for habitual reasons I usually use -R. No idea why.

lubutu• 14 years ago

In POSIX -R is the recursion flag for ls, cp, rm, etc. So if you want to recurse, -R is probably a safer bet than -r. (ls -r just lists in reverse.)

adavies42• 14 years ago

ever work on a non-gnu box? (old solaris, *bsd, stock os x, etc.) there's a decent chance they don't have -r at all.

freehunter2• 14 years ago

When I read the first post in the chain, I thought "well, it has to be a bug". Then I read the second post... wow, it was intentional. I've used -r and -R both in the same script to do the same thing, just depending on how I was feeling that day. Now I'm afraid to update.

I don't think distros should be afraid to break compatibility with main if main is making a change that makes no sense. Breaking essential and classic *nix functions defeats the purpose of CLI utilities.

pif• 14 years ago

> I don't think distros should be afraid to break compatibility with main if main is making a change that makes no sense.

Hi, I don't agree. Different versions of a tool showing different behaviour for the same option is enough for me; the same version of the tool showing different behaviour for the same option _depending on the distribution_ ... I feel that's too much!

0x02• 14 years ago

Yeah, this sounds really misguided. If they change/add functionality, why not move that to a new option? Those who want the new way can use the new feature, and scripts won't act differently depending on which version is installed.

New features should get new options; they should not move old features to new flags and put the new features on the old flags. Nuts.

rmc• 14 years ago

Increasing the number of options and switches has downsides, it makes the software more complex to use.

cschramm7• 14 years ago

I've always used -r before, since it saves a key press. ;)

But I started using -R today, in the middle of a WTF event while inspecting some Python files, digging up a Debian bug report, inspecting the upstream change and posting this to HN.

I still think it's not the best idea for Debian (or any other system) to break compatibility with upstream, since this will lead to different behaviors on different systems, not only depending on version and vendor (GNU or *BSD). But of course, it could also lead to the change being reverted, which would be welcome (guess nobody relies on the new behavior yet).

The POSIX argument is definitely valid, but nevertheless GNU's grep has always supported -r and BSD versions do as well (although OpenBSD's man page reads "This implementation supports those options; however, their use is strongly discouraged."), so it just unnecessarily breaks existing stuff.

_delirium• 14 years ago

-R is the POSIX-standard flag for recursive grep, so that would be worse to change imo. It's also the flag used for recursive grep by the BSDs (some of which do support '-r', but only as a deprecated historical option... OpenBSD calls it "strongly discouraged").

eschulte• 14 years ago

Thanks for these important missing bits of information, in light of these I think the change sounds much more reasonable.

/me updates rgrep alias to use -R

ineedtosleep• 14 years ago

Currently -r and -R are essentially the same flags, correct? And this new change would make -R and -r two different operations?

jasomill4• 14 years ago

I've always use -R, presumably because it's consistent with other commands like ls, rm, and cp (and consequently get annoyed by tools like scp that only support -r, and zip, that use -R for something else entirely); I wasn't even aware that -r was supported until I saw this.

emmelaich• 14 years ago

I've only used -R, because of it's posixness. Finger memory from adminning Solaris.

Not to say that this is a good or bad change.

-R matches the -R in chown/chgrp

They should really introduce -r for those two to NOT follow symlinks. Following symlinks for those is bad.

js23• 14 years ago

Here's the change and it's justification:

http://git.savannah.gnu.org/cgit/grep.git/commit/?id=c6e3ea6...

Change -r to follow only command-line symlinks, and by default to read only devices named on the command line. This is a simple way to get a more-useful behavior when searching random directories; the idea is to use 'find' if you want something fancy. -R acts as before and gets a new alias --dereference-recursive.

Personally I think breaking compatibility for this change was a poor decision.

akkartik2• 14 years ago

That link doesn't work. I thought at first that it was incorrectly copied, but you can see it here:

http://git.savannah.gnu.org/cgit/grep.git/log/?qt=grep&q...

And clicking on it says 'No repositories found'. Anybody else have this problem?

cschramm7• 14 years ago

Yes, seems like Savannah can't completely handle HN ;)

js23• 14 years ago

Alternate link:

http://repo.or.cz/w/grep.git/commit/c6e3ea61d9f08aa0128a0eb1...

(Yea for the "D" in DVCS.)

Also, the corresponding bug which has additional discussion:

http://savannah.gnu.org/bugs/?17623

cschramm7• 14 years ago

The commit details are back now. :)

a3_nm3• 14 years ago

Relevant thread on the ML:

https://lists.gnu.org/archive/html/bug-grep/2012-03/msg00028...

morsch• 14 years ago

the idea is to use 'find' if you want something fancy

I don't get it, can someone explain? It also appears in the ML thread linked elsewhere, and I don't understand it there either. (I understand the reasons for the change in behaviour, just not this specific line.)

bluetech• 14 years ago

find is essentially the command line tool for traversing the file system hierarchy recursively. It's got a million options, e.g. whether to follow symlinks, depth, file/directory/device/.., user, group, change time and many more. When you use grep -r, it only does the most common thing which is to just go over all the files. If you want something more "fancy", you can use find, e.g.:

  $ find -name '*.c' -type f -exec grep printf '{}' +

JoachimSchipper• 14 years ago

Remember: GNU is not UNIX.

m0skit0• 14 years ago

Backwards compatibility is an evil illness that sometimes must be broken. It's for the good of evolution. I praise engineers that make such decisions, even if they are unpopular.

cturner• 14 years ago

You'll feel differently when you inherit responsibility for some god-awful mess of shell and perl that nobody properly understands that used to Just Work but doesn't after an innocent upgrade.

TillE• 14 years ago

When there's a clear benefit, that's great. Feel free to scrap backwards compatibility when there's significant progress to be made in doing so.

This feels a lot more like a "color of the bike shed" choice. There are use cases where the new behavior makes sense, sure, but there are also plenty of cases where the old way is better. This isn't an upgrade, it's a lateral move.

tinco2• 14 years ago

So we have to continue with an ugly bikeshed for the rest of eternity? I think it's very important that whenever it is decided that objectively one way of doing it is better than another way that eventually that way finds itself to be the way it is.

You basically imply that this change is not big enough to warrant a break with backwards compatibility, but small discontinuities and hacks add up. If thinks like this aren't fixed every once in a while the system will be ridden with inconsistencies.

Besides, even though it hurts when you've inherited some crazy unreadable code that utilizes some obsoleted functionality I think it is always positive for the code quality when the chaos monkey comes around and breaks something.

edit: the downvote button is to indicate I am detrimental to the discussion, the reply button is for when you disagree with me :)

droithomme• 14 years ago

Grep is used in hundreds of millions of scripts ranging from mundane rarely used things to install scripts to critical system functionality run every few minutes.

It's not acceptable to break the default way it works under any circumstances.

jasomill4• 14 years ago

I agree in principle, though to be fair most UNIX utilities have so many incompatible variants that striving for maximum backwards compatibility often does more harm than good (like producing options that do entirely different things in the presence of other options, in the absence of a leading hyphen, etc.). Practically speaking, sysadmins generally use more consistent tools (e.g., Perl) for nontrivial cross-OS things once this becomes an issue, and, as a developer, it's not clear to me that build-time dependence on, say, Perl or Python is any worse than depending on GNU versions of basic UNIX tools, the existence of which tends to only be a safe assumption on Linux. On non-UNIX platforms, this is an even bigger issue: I'd certainly rather recommend ActivePerl or the latest binary Python 2.7.x release from python.org to the average Windows developer than any of SFU, Cygwin, or MSYS (and I say this as someone with a strong UNIX background who works with both Windows and Windows developers on a daily basis).

guard-of-terra• 14 years ago

grep doesn't need any more evolution. grep is the final result of the evolution.

larrik• 14 years ago

The biggest problem I see is: who the heck would expect a .12 release to break compatibility in an ancient and solid piece of software?

sneak• 14 years ago

Why is grep being modified at all? It is clearly not broken.

Aglet• 14 years ago

To quote a recent Coding Horror post, its bugs are now Common Law Features, and must be supported.

cschramm7• 14 years ago

Another thought: Since rgrep is an alias for grep -r and grep -r has changed, the complete rgrep tool has changed and this can _not_ be fixed by using the "right" switch. Hence rgrep is useless for scripting now.

pooriaazimi2• 14 years ago

This is a serious question: Why would you want to use plain old "grep" instead of "ack"[1]? Of course, other than the fact that it's on all machines. Why would you use it instrad of "ack" on your own machines? That's an honest question, I'm not starting a flamewar... The highlighting and filename/line# by default is the killer feature for me.

Edit: Come on. Downvotes for this? Honestly... It's HN, not StackOverflow. You don't mark questions as "off-topic" unless they're trolling...

[1] http://betterthangrep.com

audiodude• 14 years ago

I love ack, I've never looked back.

MattJ100• 14 years ago

The main reason to use grep is portability/availability. While a modern improvement to grep (yes, I'm an ack user and fan too) you can't simply depend on it in portable scripts and such.

grep is defined for POSIX after all: http://pubs.opengroup.org/onlinepubs/009695399/utilities/gre...

I also suspect (without looking) that there are obscure (but occasionally useful) grep features that ack doesn't support.

greyboy2• 14 years ago

That's a fine recommendation for machines that you solely work on, or have complete control over. However, it's good to be able to work with the standard toolset if you frequently work on a variety of remote machines (where it's quite common that you cannot install such things, due to permissions or policies, and want to get started working before attempting to download/install a bunch of custom binaries).

Edit: MattJ100 made a more clear point while I was responding.

pooriaazimi2• 14 years ago

Thanks for the response. However, a nice thing about ack is that it's not a binary (necessarily) - it's a perl script and can be used without sysadmin permissions: http://betterthangrep.com/install/

The other points are quite understandable and correct though. It's always best to at least be familiar with standard tools, even if you want to use an slightly different version for yourself.

eliasmacpherson• 14 years ago

ack has ignore built in for certain filetypes. Then combined with my shell expanding * to just the files and directories at the current layer, ack will not find certain things that grep -R string * will.

a3_nm3• 14 years ago

ack-grep is a good tool to grep through files. grep is more than that. When I need to grep in a pipeline, I probably don't want ack-grep's bells and whistles.

One example of a useful feature of grep: "grep --line-buffered" to grep with no buffering in a pipeline.

benatkin• 14 years ago

It's called ack. Ubuntu gave its package and executable a different name because it conflicted with something far less notable (poor choice IMO). I suggest adding an a alias or a symbolic link so you can call it by its proper name when typing in commands.

jasomill4• 14 years ago

Because it motivates me to avoid revision control systems that insist on crudding up every directory in my source tree?

pbhjpbhj• 14 years ago

>Why would you want to use plain old "grep" instead of "ack"[1]? //

First time I've heard of it. Thanks. Only been grepping my way around for the last dozen years or so ...

ghshephard• 14 years ago

I've been a systems administrator of one form or another for 19 years, spend 2-3 hours a day on the CLI, and I've never heard of ack. Depending on the time of day (and platform) i might use grep or egrep. I hopped onto one of the random ubuntu boxes I own, did a quick "apt-get install ack; man ack" - here is what it said:

"ACK is a highly versatile Kanji code converter. ACK can do reciprocal conversion among Japanese EUC"

This is probably why I don't use ack - never heard of it, not available on any system I use, and the dpkg repository has something that has nothing to do with grep.

Enough of an answer?

ksherlock• 14 years ago

debian installs it as "ack-grep".

csense• 14 years ago

> Why would you want to use plain old "grep" instead of "ack"?

1. Because all the cool kids are doing it.

2. Because I hate Perl.

koenigdavidmj2• 14 years ago

Why is your Perl hate relevant? It can be written in COBOL for all I care; I'm just using a tool to find files. I don't spend my days browsing through the source code of random tools.

snorkel• 14 years ago

I also decided that sed no longer supports regexes because I said so. And ls -l will now shows a list of print jobs instead of files because maybe that's what you meant. ... oh and ping no longer supports IPv4 because I want everyone to adopt IPv6 immediately because I said so.

rwos• 14 years ago

It's funny how one of the relatively few design errors in Unix shells now indirectly comes back to haunt us. Recursing is built into pretty much every command that can handle multiple files - which very strongly suggests it should have been made a feature of glob (or the shell).

I think it's a bit sad that those "big-picture" features in unix are treated as if the were written in stone.

nvarsj• 14 years ago

This is one of the great reasons to use zshell. :-)

  grep -in **/*txt "sometext"

GNU grep 2.12 changes behavior of recursion options, breaks existing scripts

Related Stories