A cron pitfall that will probably snare you at least once in your career

http://www.reddit.com/r/raldi/comments/i8z20/frustrating_unix_pitfall_of_the_day_esoteric_cron/

by raldi • 15 years ago

136 67 15 years ago

jerf• 15 years ago

Asked there, but I'll tack it on here too: What is the purpose of that absurd-seeming restriction on file names? If you can drop arbitrary executables into directories on the file system, you've already "won", so I assume it's not security. I've read the run-parts man page, and that just pushes the problem back one level. If it's just to avoid running old files, that's an absurdly large hammer to hit that problem with.

raldi5• 15 years ago

My guess is they wanted to avoid running .dotfiles and #emacsbackup~ files and threw out the baby with the bathwater.

pyre2• 15 years ago

Also '.bak' or '.old'

dexen2• 15 years ago

Are #emacsbackup~ files executable by default? If not, that's a very weak reason.

(honestly I dunno; have never used EMACS yet)

cperciva2• 15 years ago

I have no clue why run-parts does this; but BSD periodic doesn't.

mikemaccana• 15 years ago

Yes, it's not a bug in Vixie Cron (which I think BSD cron may use) but in the Linux distros (who are responsible for adding run-parts).

.py, .pl, .sh and (as people realize they'are actually writing bash scripts) .bash are standards people actually use. Having them break in what amounts to an include dir is bad mojo.

insipid2• 15 years ago

According to the summary on the Ubuntu bug: "Essentially, we don't want to allow a '.' because of .dpkg-old and .dpkg-dist files."

Is the implication that you can have .dpkg files in cron folders that you wouldn't want to try to execute?

https://bugs.launchpad.net/ubuntu/+source/debianutils/+bug/3... http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=68561

nbpoole6• 15 years ago

I posted a link (http://news.ycombinator.com/item?id=2701527) to a Debian bug that explains the rationale (aside from the historical behavior argument) a little better: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=308911

tvon2• 15 years ago

I'd love to hear the rationale behind the dot restriction.

chrisjsmith2• 15 years ago

Been there done that. It's very annoying indeed. I wish things were slightly less finicky in *NIX platforms as a rule. They need a serious de-crufting session (plan 9 looks very much less crufty).

bshep• 15 years ago

I've been hit with this problem before and had never found an explanation why some scripts worked and some didn't.

I eventually just added it to the crontab of a user and that was that, but I'm so glad to finally know WHY.

This is such an arcane restriction and its really hard to debug since there are no errors anywhere.

chrisjsmith2• 15 years ago

The lack of errors was annoying and counterintuitive. I like the usual UNIX mantra of blow up noisily if an assumption is made but it just doesn't happen there.

yock2• 15 years ago

I think I'm missing something. The comments that appear as if they should contain the answers contain nothing. Have the answers been removed? Does Reddit's [spoiler] tag do fun javascript-y things that don't work on IE8? Or am I so obtuse and unobservant that I'm missing something obvious?

nbpoole6• 15 years ago

Click and drag over the text with your mouse and you should be able to see it.

bmunro• 15 years ago

That was the first thing I tried. It doesn't work for me in Chromium 12

randombit• 15 years ago

They show up as a hovertext in FF

gcb• 15 years ago

useless on android

mthomas• 15 years ago

You simply highlight the blocked text to reveal it. The text is the same color as the background.

yock2• 15 years ago

That was the first thing I tried, to no avail. I was able to see his response by going to his profile and hovering over the word "spoiler" in his comment stream. The word "spoiler" doesn't appear for me in the threaded comments and clicking through the word "spoiler" results in a 404.

Very strange.

GrooveStomp• 15 years ago

Yeah, it wasn't working on my Android phone either. Had to wait to get into work and try it with Chrome.

nbpoole6• 15 years ago

I found an older bug report for Debian that sheds a little light on why this behavior is useful: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=308911

    I can understand not changing the behaviour wrt historical
    precedent: I have been told that people used to rely on the
    ignore-dot mechanism to disable things (e.g. mv foo foo.disabled).

At the end of the report, you can see they added the --regex flag to allow for user-defined filename validation.

raldi5• 15 years ago

They could at least log the skipping, by default.

jsprinkles3• 15 years ago

Shhh! Don't make administrators' jobs easier! How else will we justify our insane salary and benefits?

Look at the bright side: this is a few days of debugging and difficult to explain to the people signing your check. Aren't you glad it's there now?

ars2• 15 years ago

Considering how often these files are run that would make for quite a noisy logfile.

ck2• 15 years ago

I don't believe CentOS distro has this problem?

I have plenty of scripts and php and such running in crons all over the place.

jwr• 15 years ago

This bit me as well. It is particularly annoying, as it takes forever to actually find the problem.

In addition to that, the restriction is silly and utterly pointless.

mseebach• 15 years ago

run-parts is used in many places on linuxes (anywhere you see a .d directory) so this is not a cron pitfall.

jsprinkles3• 15 years ago

You're right. Upstart does this too (if you rename to .disabled in /etc/init the job will not be run).

js2• 15 years ago

Frustrating Unix pitfall of the day: esoteric cron rules

This isn't a Unix pitfall, it's a Debian pitfall. Further, it's not inherent to cron, but to the /etc/cron.{hourly,daily,weekly,monthly} concept as implemented by Debian.

Scripts in these directories are run via /usr/bin/run-parts, which is invoked via /etc/crontab. It is Debian's run-parts that imposes the naming restriction. Red Hat's run-parts doesn't have this restriction (go take a look, it's a shell script).

tedunangst• 15 years ago

Thank you for that clarification. I've been running shell scripts named .sh all the time without trouble. Of course, I run them from crontab. I'd never even heard of this /etc/cron.* thing.

sliverstorm• 15 years ago

It's somewhat nifty, I suspect it originated as a way to allow new packages to easily add jobs to cron if they need

hollerith4• 15 years ago

>This isn't a Unix pitfall, it's a Debian pitfall.

And that is an example of why I do not run Debian any more: (1) it assumes that I have nothing better to do than learn all of the details of the Debian system and (2) as soon as some complication like this dots-in-filenames thing makes it into stable, it is very difficult to remove because most Debian maintainers consider it-might-break-existing-code a winning argument.

ramidarigaz• 15 years ago

What do you use instead?

barrkel• 15 years ago

"It might break existing code" is usually a winning argument. It takes a very strong opposing argument to beat it. If you don't believe in this, I don't want to use any of your code.

pyre2• 15 years ago

On the other hand, CentOS 5 has run-parts, but it doesn't have a --help option or a manpage. The Debian one does.

raldi5• 15 years ago

Hey, when you only get 80 characters for the headline, certain things have to be glossed over.

javanix• 15 years ago

It is kind of amazing how much this explains.

I thought I was doing well when I finally learned to remember the "no cron files that don't include an empty newline" rule.

xbryanx• 15 years ago

I've been using Jenkins to do every single thing I used to do in cron these days. Lots easier to visualize execution, and catch potential errors. More overhead, yes. Saved me tons of time, yes.

geuis• 15 years ago

Why the devil can't I see certain text? I'm not a user of reddit so is this some kind of joke? I'm reading this on my iPhone.

whiskers• 15 years ago

Also on an iPhone. The reason is they're using a 'spoiler' tag that hides the text until it is selected. Unfortunately we can't do that...

city41• 15 years ago

Tap the text and it will become visible, then it navigates to reddit.com/s (no idea why). Then click back and the text will still be readable (just confirmed on my iPhone 4)

technomancy• 15 years ago

Another fun one is that if your file doesn't end in a newline, cron will silently ignore the last line! Lost a couple days of production data at my last job due to that "feature".

oasisbob• 15 years ago

IIRC, OpenBSD has some similar issues with PF rules, and perhaps other configuration files as well.

I can't remember if it refuses to parse the file or ignores the last line, but it's not a fun thing to bump against.

there• 15 years ago

what editor are you using that doesn't add a newline?

res0nat0r• 15 years ago

Since everyone should hopefully know /etc/cron* is a nonstandard cron directory, and an addon from Debian, it would be hopeful that you would also have discovered the relevant documentation as why this is, ie run-parts(8), like mentioned.

The manpage states you can pass --test <dir> and it will show you all the scripts it would have run. This should have been something done before any assumption was made as to this working off the bat.

On my systems now, or old Solaris machines I used to admin, I would always do an "at now <script>" to make sure there were not any $PATH issues before putting the script in production. I've always tested to make sure the generic skeleton of the script would work before assuming anything...

A cron pitfall that will probably snare you at least once in your career

Related Stories