Common Bash Pitfalls

http://mywiki.wooledge.org/BashPitfalls

by dunk010 • 17 years ago

60 25 17 years ago

mywiki.wooledge.org

jriddycuz• 17 years ago

Thanks for this post. I'm new to doing bash scripting and this is very helpful.

dunk010• 17 years ago

I've got the O'Reilly bash book, it's well worth getting.

fungi• 17 years ago

the only university text book i've held on to.

ilyak• 17 years ago

Most of those aren't specific to bash.

hernan73• 17 years ago

Correct -- many of these are also problems in Bourne or Korn shell.

I guess by virtue of being the default shell on most Linux distros, "bash" is now seen as a synonym to "Unix shell programming".

imd• 17 years ago

I see it more as: people who have problems while working with Bash will go to the Bash wiki, even if the mistake they're making could be made in other shells.

barrkel4• 17 years ago

The only one I didn't know about is the problem with:

     echo "hello world!"

I'm surprised history expansion could reach inside the quotes.

mqt• 17 years ago

You need to use single quotes. !, $, `, \ are expanded inside of double quotes.

barrkel4• 17 years ago

I know about variable and command substitution and escaping, it makes sense that those expand inside " but not ' - but history expansion, something seemingly inherent to the interactive REPL, seems out of place here.

WilliamLP• 17 years ago

Is Bash scripting still relevant today, other than working with and understanding legacy code? Why wouldn't I use something like Python or even Perl instead?

CrLf• 17 years ago

Shell scripting is as relevant as it ever was.

Python or perl scripts are fine if you are doing more complex and "algorithmic" stuff, but for automating system administration tasks, I often find that shell scripts are simpler and more concise.

But shell scripting isn't just "scripting" (as in ".sh" files with commands). A proficient unix user/sysadmin will often build one-shot commands using features other people thing are only useful in batch scripts. Actually this is something I find baffles both the Windows-types and the people that solve every problem with "perl -e".

nailer• 17 years ago

You're right about people needing to understand bash and Unix commands - I've seen two 20+ line Perl scripts that run the equivalent of 'date -I' and 'nc' respectively.

That said, I believe bash is somewhat limited for system administration - which I know is very odd amongst system administrators - but please let me explain why.

Using the shell on a Linux based OS in 2009, you miss out on:

* filesystem events (inotify)

* hardware events (dbus)

* config for programs with tree structured configuration (lxml)

* config for programs with sqlite based configuration

* RPM / yum beyond what their command line apps expose (and perhaps dpkg/apt too, but I need to investigate this further).

...as the commands / shell function libraries to handle these are often immature compared to the equivalent APIs. As a result, you get sysadmins causing unnecessary load by polling a file repeatedly at intervals (eg, via a cron job) rather than letting the kernel tell them when their file has changed.

I'm hopefully giving a talk about using richer languages for system administration at PyCon 2010 - I'd love to hear any thoughts or opinions on these matters fron HN readers.

jsrn• 17 years ago

Apart from the reasons CrLf was pointing out, here is another one: The shell (and a set of utilities) is standardized - it's part of the POSIX standard - this is important for some organizations. If you program in the subset of bash that conforms to the standard (enforcable with bash --posix) you are programming against an interface that will likely be more stable over the coming years than Python or Perl (think Python 2->3, or the long awaited Perl 6).

Standard Shell Command Language: http://www.opengroup.org/onlinepubs/9699919799/utilities/V3_...

As always, it's a tradeoff, i.e. the best solution depends on the problem and requirements. Many problems shouldn't be tackled with the shell, for sure...

jockc• 17 years ago

I do it every day using Cygwin. Certain tasks are so much quicker when you can for/while/do/cut/grep/sort/uniq/wc/>< and so on.

hernan73• 17 years ago

In my experience, the main advantage of bash is conciseness for certain tasks. I don't have much experience with Python, but I have found the shell (and associated command) less verbose than Perl in certain cases.

Obvious things like "find" and "grep", but also things like doing things with the output of a command. In Perl, you need to open the command as a file handle, which takes a couple lines of code. In bash, you just use "|".

I wouldn't write a multi-thousand program in bash, though. Not again, anyway...

gcv• 17 years ago

Good question. I can think of two tasks for which I routinely use the shell, and wouldn't really prefer one of the newer scripting languages:

1. Customizing my interactive shell, although I personally favor zsh over bash. I write shell functions and automate PATH setting and discovery where possible, so knowing how to script the startup file helps.

2. Startup scripts for software I distribute. Perl or Python would work, but then I would have to be careful about which version of the language to use. With /bin/sh, as long as I use purely POSIX constructs (which means no arrays, unfortunately), scripts work anywhere.

nas• 17 years ago

The quoting rules in the Plan 9 shell rc are much saner. Bourne and C shell quoting rules are a recipe for trouble. Sloppiness (e.g. leaving out quotes, using the wrong kind of quotes) works about 99% of the time which leads you to think your code is correct.

The big difference is that rc does not expand the result of variable expansion. I've been planning to write a rc derivative that had job control for a long time now. So far, I've never had enough spare time.

10ren• 17 years ago

It's the problem of nested DSLs (sed, tr, globing, bash's own control commands etc): you need to quote (escape) each DSL's meta-characters like crazy.

It's a scary, horrific nightmare. Sure, you can learn all the intricacies, but what a waste of your cognition! There must be a way to do it with clean abstractions (assuming the unrealistic liberty to redesign all unix commands). But... I have a suspicion that the horror is intrinsic to the problem, and bash strikes a legitimate trade-off between brevity and safety. It would take some work to be sure about this.

gloob• 17 years ago

Based on no evidence whatsoever, my gut tells me that trying to set up a tidy system of abstractions to clean up the problem would produce something closer to a programming language than a shell language. On the other hand, it's possible that exposure to *nix has simply impoverished my imagination when it comes to things like this.

Common Bash Pitfalls

Related Stories