Bash scripting cheatsheet

https://devhints.io/bash

by GordonS • 8 years ago

137 30 8 years ago

devhints.io

egwynn4• 8 years ago

The very first example (echo "Hello $NAME!") won’t work if `histexpand` is set, which I believe is the default.

ben5096• 8 years ago

It works if the ! is quoted, as in the example.

egwynn4• 8 years ago

That is incorrect. From the man page:

    If enabled, history expansion will be performed
    unless an !  appearing in double quotes is escaped
    using a backslash.

blueflow• 8 years ago

Also, '< file.txt | ...' does not work as well. Not sure if its well researched.

egwynn4• 8 years ago

Yeah my eyebrow went up at that one too…

ben5096• 8 years ago

Which is surprising since "research" here means pasting it into a terminal...

Maybe they meant `cat file.txt | while ...`? Even then, you never want to do that when you're reading because elements of a pipeline are run in subshells. The usual idiom is:

    while cond; do
       thing
    done < input > output

It's also the problem with cheatsheets: the underlying idea in bash is that compound statements are like mini programs, so they have file handles that can be redirected. That doesn't come through a cheatsheet very well.

gist• 8 years ago

We used to use things like this way back. In fact I would purchase these laminated sheets full of hints on various topics (c, unix, bash) and so on.

The thing is I actually find it quicker to simply use my own notes today and/or google anything I need to know when I need to know it.

ben5096• 8 years ago

Note that [[ is actually a command/program that returns either 0 (true) or 1 (false).

This is true for the [ command, which is the same as `test`, but the [[ ]] syntax is parsed directly by bash, as is (( )). In particular, [[ ]] is not subject to whitespace splitting, so you don't need to quote most variables.

They mention $(( )), but don't go into (( )), which gives you very nice integer arithmetic and logic:

    for (( x = 0; x < 5; x += 2 )); do
       echo $x
       (( y = x * 10, z = y + 2 ))
       echo $y $z
    done

chubot• 8 years ago

Yes I wrote about this on my blog:

[ Is a Builtin, But [[ Is Part of the Language

http://www.oilshell.org/blog/2016/10/12.html

I didn't know that before I started implementing a shell :)

I knew that [ could be a builtin or /usr/bin/[, but not about the difference between [ and [[.

ricardobeat2• 8 years ago

My bash scripting cheatsheet after maintaining a couple tools for 4+ years: use system Ruby / Python or anything else instead, really. Once over a hundred lines of code it becomes a massive pain to work with.

juddlyon• 8 years ago

Python is fantastic if it requires functions or anything over several lines. I'm not a Python dev but I can very quickly find standard library examples or well-documented packages.

Perl/Ruby are too clever for me.

tracker1• 8 years ago

I've been doing similar with node+shelljs for similar reasons. Especially with Windows being used by a lot of devs where I work.

This cheatsheet is still a nice to have though.

RcouF1uZ4gsC• 8 years ago

For Python, I have used plumbum https://plumbum.readthedocs.io/en/latest/ which basically makes Python as terse as bash for common use cases and greatly simplifies the complicated stuff.

h1d• 8 years ago

Shell scripting has a place and script languages aren't always the answer for improvement. For many tasks it can even be more verbose but logic crafting can be easier.

You should rather look for languages like fish, which makes shell scripting feel better than drop it altogether.

oxguy3• 8 years ago

The biggest benefit to Bash scripting over a friendlier language is that Bash is guaranteed to just work on pretty much every machine. Bash is on virtually every *nix machine (except maybe embedded devices), and it's extremely resilient to time. Looking through the change history (https://wiki.bash-hackers.org/scripting/bashchanges), I'd wager that 99% of scripts don't use anything that wasn't in Bash 2.05b, which came out in Jul 2002. Anecdotally, the "newest" feature that I've ever used is regex support in conditional expressions, which was added in 3.0-alpha (Aug 2004).

Ruby and Python, meanwhile, are not quite so universally installed as Bash, and they have had all sorts of major breaking changes since 2004. And even if they are installed and version-compatible, they still might not work. Countless times have I seen a misconfiguration with Python 2 vs 3 or RVM or the PATH variable or whatever else cause errors, but I have never run into a scenario where a Bash script didn't just work.

All that said, I'm totally with you that Bash scripts should be short and sweet. If you're doing something with over 100 lines, Bash's downsides overtake its benefits.

roryrjb• 8 years ago

It's true that bash can be installed on any Unix-like OS, but so can Python, Perl, etc, but it's not available by default, on OpenBSD there's pdksh, FreeBSD tcsh iirc, and on MacOS I believe the version of bash is really old due to some previous license change.

Edit: I read your post quickly, you addressed the old bash scenario, but still in my opinion I would always, and do target POSIX shell for maximum cross-platform compatibility and that is definitely available on Unix-like OSs by default.

ricardobeat2• 8 years ago

That’s true, and it’s the reason the tools I mentioned were written in Bash in the first place.

Python 2/3 was a pain, Ruby had very disparate versions available. But now, a few years later, you can assume at least Ruby 1.9 nearly everywhere, and OTOH still can’t use any of the Bash 4 features. There also quite decent cross-platform packaging solutions for python, ruby, node, go, crystal, mruby that make it really hard to choose bash for any serious project.

oweiler• 8 years ago

While Bash itself is portable, the commands Bash scripts glue together are often not.

Domenic_S• 8 years ago

Related, I love to send my bash scripts though https://www.shellcheck.net/

Karupan• 8 years ago

Shellcheck is awesome! Been using it for about a month and it’s great to understand certain behaviour in the script, even if I don’t end up fixing every single warning.

mistrial9• 8 years ago

.. looking at >1000 LOC bash project this week, with autotools install to make them go.. it uses 'SOURCE' as an include mechanism.. runs FAST and uses lots of whitespace, but no one on earth but the original author wants to touch it

ben5096• 8 years ago

If you want to port such a project:

Start by moving code into bash functions if it's not already doing that.

Declare variables with `local` to reduce the amount of global state.

Once you have code in functions, you can factor code into independent scripts. Anything that runs in a subshell can move into its own script, so anything:

1. in a pipe | line

2. running in the background&

3. in a ( subshell )

4. that doesn't modify global variables

You can then work on those scripts piece by piece.

zimbatm2• 8 years ago

0. run shellcheck and fix all the warnings

Using shellcheck will give the developer more confidence in their code, and also learn through the feedback it gives

karmakaze• 8 years ago

My newly forming pet peeve are cheatsheets that are long scrolling documents with way too much whitespace.

Cheatsheets are dense, well formatted items that either demonstrated a thing or reminded you of a thing. Ideally they would be printable in either letter or A4, possibly double-sided.

We need a different word for what this is. Tips/hints blog post?

gshubert17• 8 years ago

Plus this page doesn't print nicely, either to paper or PDF.

zimbatm2• 8 years ago

Here are two lesser-know tricks that can be quite handy:

Using () instead of {} for the functions body allows to isolate it:

    foo=5
    bar() (
      cd /
      foo=7
    )
    bar
    echo $PWD # not / but the current directory
    echo $foo # still 5

Bash 4.0 can splat array of arguments back into a valid bash string with @Q. This is super useful when you need to stop using arrays, like with `bash -c`

    ary=("arg1" "arg 2")
    echo "${ary[*]@Q}"
    # outputs: 'arg1' 'arg 2'

iClaudiusX• 8 years ago

A few things I would add:

brace expansions

  combine for combinations

    {a..c}{1..3}  # a1 a2 a3 b1 b2 b3 c1 c2 c3

  step size

    {0..10..2}  # 0 2 4 6 8 10

mapfile to read lines from file to array

  mapfile -t arrayname < filename

    -t     # remove trailing newline for each element

    -u FD  # read from file descriptor FD

    -s N   # skip first N lines

    -n N   # read at most N lines

    -O N   # start populating array at index N

array quoting

  "${array[@]}"  # expand all elements, individually quoted

  "${array[*]}"  # expand all elements, group quoted

translate number from base to decimal for base in (2,64)

  $(( base#num ))

Bash scripting cheatsheet

Related Stories