Less Commonly Used Unix Commands

http://www.danielmiessler.com/blog/collection-of-less-commonly-used-unix-commands

by danielrm26 • 12 years ago

188 92 12 years ago

danielmiessler.com

JetSpiegel• 12 years ago

Worth it for sshfs alone.

addisaden• 12 years ago

in combination with iftop

vacri2• 12 years ago

A friend of mine once commented "If you need to use sshfs, you've already lost"

laumars• 12 years ago

Maybe your friend is thinking too much about enterprise set ups. I have no need for sshfs at work, but for my personal servers that's a different matter. I find it invaluable to have the convenience of being able to have an NFS-like share over a simple and secure SSH socket. Yes you could literally run NFS over VPN or an SSH tunnel, but sshfs is just so much more convenient.

So really it just depends on what you're trying to do as to the practicality of sshfs. But then I guess you could argue that is the case for nearly all the commands that many sysadmins have flamewars over.

robinhoodexe• 12 years ago

Nice, although many of them are used quite a lot. Or maybe that just me.

arca_vorago• 12 years ago

One that I have recently been using more and more is time, but don't forget there is a difference between bash's "time" command and /usr/bin/time . I use both.

The majority in the list seem pretty common, but I guess with more and more gui dependents some people just never spent days wading through man pages or info coreutils, so still useful.

Blahah• 12 years ago

/usr/bin/time is really useful, because it has the-v flag which tells you memory and processor usage.

Aloha3• 12 years ago

tr and lsof

ben336• 12 years ago

vim and xargs are less commonly used commands? Seems to be a mix here of small unix-y utilities and larger programs (like vim and tmux). Good stuff to know though.

dctoedt• 12 years ago

FTA: vim: attack yourself

Emacs user?

holyjaw• 12 years ago

Nah, he's a vim user IIRC. He has a really good vim tutorial on his site [0].

[0] http://www.danielmiessler.com/study/vim/

JadeNB2• 12 years ago

> FTA: vim: attack yourself
Emacs user?

As holyjaw mentions (https://news.ycombinator.com/item?id=7253126), this appears to be a vimmer's self mockery, rather than an Emacsy attack:

http://www.danielmiessler.com/blog/categories/vim

Aloha3• 12 years ago

Emacs User at Work:

http://fc04.deviantart.net/fs71/i/2011/029/8/3/emacs_user_at...

oxymoron• 12 years ago

I found jq the other day (http://stedolan.github.io/jq/), which is a commandline utility for manipulating json data, resembling awk.

Considering the number of API's that return json data these days, it's simplified my life a great deal. It's proven especially useful for automation shell scripts, when working with things like the aws command line client that return all results as json. Usually I'm looking to extract something like a single field to use as input for a following command, and jq makes it really easy. No more inline python or awkward sed regex's. :)

It's available using apt-get on debian/ubuntu, but that version is out of date with the online docs. It's rather trivial to build from source though (it has no external deps).

brokentone• 12 years ago

+1 for jq. Lifechanging. I had to build it on CentOS6, but as you said, super easy.

jalkjaer• 12 years ago

xmlstarlet does similar things for xml

shadowfox• 12 years ago

On a similar note, there is also csvkit [1] for csv files. It is not quite as full featured as I had like, but it is easier than manipulating csv content directly via cut/column/grep/sed.

[1] http://csvkit.readthedocs.org/en/latest/index.html

pessimizer• 12 years ago

try csvfix: http://csvfix.byethost5.com/csvfix15/csvfix.html

hrez• 12 years ago

With aws cli you don`t need jq. Read on --query option. http://docs.aws.amazon.com/cli/latest/userguide/controlling-...

Mister_Snuggles• 12 years ago

wget: get w's

fetch is also useful to get w's.

VLM• 12 years ago

Site's non-responsive now, wonder what the list was?

I'll vote for "yes" which at least seems intuitively useful in a "yes | rm -i something" kind of way. Well that specific example is useless... Also over two decades ago if you obtained access to someones account it was considered highly amusing to end their .profile with a call to /bin/yes. Oh and before the newfangled /dev/urandom (or /dev/zero) we used to redirect "yes" out to media to wipe them or test overall system thruput.

Now for one thats truly obscure and ripe for abuse because no one ever runs it anymore, try RMT the remote magtape manipulator.

JadeNB2• 12 years ago

I'll vote for "yes" which at least seems intuitively useful in a "yes | rm -i something" kind of way.

I use it when I want to compile a LaTeX document and ignore errors:

    $ yes ^M | pdflatex document

(useful because TeX tools hate accepting command-line options).

mzs• 12 years ago

    column: create columns from text input
    tr: translate/substitute/delete input
    join: like a database join but for text
    comm: file comparison like a db join
    paste: put lines in a file next to each other
    rs: reshape arrays
    jot: generate data
    expand: replace spaces and/or tabs
    time: track how long a task takes to run
    watch: execute something on a schedule in realtime
    xargs: execute something on all inputs
    iftop: visually show network traffic
    htop: show system stats more powerfully
    xxd: manipulate files in hex
    mtr: powerful traceroute replacement
    mdfind: osx find replacement that uses spotlight
    brew: osx package manager
    df: disk usage
    du: disk usage
    dig: dns queries
    host: dns queries
    man ascii: lookup your ascii
    sshfs: mount a directory through ssh
    wget: get w’s
    tmux: a better screen
    pushd: push your pwd to a stack
    popd: pop pwd off your stack
    lsof: godlike
    ncat: use this instead of nc
    fuser: kills locking processes
    vim: attack yourself
    tac: cat in reverse
    rename: change spaces to underscores in names
    open .: in osx, open finder in the current dir
    lsmod: show kernel modules
    printf: change the format of output
    timeout: execute something and kill it soon after
    disown: protect a job from disconnect
    fc: edit your last command in your editor and execute it
    sudo !!: repeat last command with sudo
    tee: send output to stdout as well
    pgrep: greps through processes
    pkill: kills processes based on search
    fmt: text formatter
    bc: an interactive calculator language
    apropos: info on commands
    strace: the uber debug tool
    man units: interesting
    pstree: shows processes in a…well…tree
    ddate: wtf
    zgrep: grep within compressed files
    zless: look at compressed files
    readlink: read values of links
    atop: another top
    split: break a file into pieces
    dd: disk writing
    ndiff: show differences in mmap scans
    ss: show apps using the internet

cellover• 12 years ago

whatis <cmd> when unsure of what a command does is quite helpful and displays short manual page descriptions, for instance some metaphysical question: > whatis time

arjn• 12 years ago

Plenty on this list are well known but there are some nice ones that either I'd forgotten about or didn't know : column, ss, comm, fc

I'd add the following to the list : printf , bc

Also, "ddate" is fun. I had no idea and just spent an enjoyable 10 minutes looking it up.

Thanks for posting this.

Groxx• 12 years ago

Since you seem to know what "ss" is, mind if I ask for a link? It's rather hard to search for :/

untothebreach• 12 years ago

http://linux.die.net/man/8/ss

banachtarski2• 12 years ago

Plenty on this list are well known...

My thoughts exactly. Time, df, du, iostat, watch, htop, wget, tmux, ....

Really?

nknighthb• 12 years ago

Most people know only the bare minimum to move files around. Not everyone has spent their life on a nix command line.

And I don't even know why you'd think iostat, watch, htop, or tmux would be well-known amongst anyone other than sysadmins and long-time nix power users. And tmux didn't even exist five years ago.

jumpwah• 12 years ago

fc, tac, sudo !!, vim, ...

INTPenis2• 12 years ago

I only recently learned about ioping, which I think would fit into that list well.

angersock3• 12 years ago

popd and pushd are commands that, once learned, I thought would change my life forever.

Turns out, not so much. :|

EDIT:

'watch mtr' is handy for keeping an eye on your network connection, and finding slowdowns.

simcop2387• 12 years ago

Using some aliases you can have pushd automatically. What then ends up useful with that is you can use ~1 ~2 ~3 ... to refer to old directories. That will change your life :)

http://unix.stackexchange.com/questions/4290/aliasing-cd-to-...

theophrastus• 12 years ago

i also tried to use pop/pushd ...tried hard. and never got it integrated into the work flow. however this guy's method has saved me a lot of keystrokes (very handy if one has a few, frequently visited, yet deeply nested, directories): http://jeroenjanssens.com/2013/08/16/quickly-navigate-your-f...

eropple2• 12 years ago

I aliased `cd` to pushd (squelching output) and `dc` to popd in my interactive shell. It's super handy to have a back button for my shell.

angersock3• 12 years ago

Oh, that's cute--I like that. :)

quesera• 12 years ago

dc(1) is also an RPN calculator, which comes in handy sometimes.

LVB• 12 years ago

Agreed. The game changer for me turned out to be: https://github.com/rupa/z

It has an uncanny knack for getting me to where I want to go with very little in the way of input.

justinmk• 12 years ago

fzf completely replaces z, and is ridiculously fast.

https://github.com/junegunn/fzf

vacri2• 12 years ago

Why do you need to 'watch' mtr? It functionally behaves as a watched tool to begin with.

angersock3• 12 years ago

Because I'm dumber than a sack of rocks, apparently. :|

davexunit• 12 years ago

Why do programs that only work on OS X always end up on lists of "Unix" programs?

Aloha3• 12 years ago

OSX is unix, in the same way that Solaris or AIX is, but I'd have split them out in a separate section for OSX specific Unix software.

chimeracoder• 12 years ago

Exactly - OS X is Unix(tm), but davexunit's complaint is that the term "Unix commands" implies "commands that will work on all/most UNIX/POSIX systems", not "commands for a single OS that happens also to be a UNIX".

wicket• 12 years ago

Agreed. Anything that is not part of the SUS or POSIX standards should be in a separate section indicating their origin/availability.

crymer11• 12 years ago

"The most common version of Unix (bearing certification) is Apple's OS X" - http://en.wikipedia.org/wiki/Unix

pjmlp4• 12 years ago

Mac OS X being UNIX certified, does not mean all Mac OS X commands are available across all UNIX variants.

It rather implies that Mac OS X supports what UNIX certification requires, plus whatever extensions Apple does on top.

This was always the main issue with POSIX compatibility fragmentation across UNIX platforms.

chc• 12 years ago

Probably the same reason Linux users would put Linux programs on such a list — it's relevant to them and probably much of their audience.

pjmlp4• 12 years ago

And then they will be lost looking for then at an HP-UX shell, as an example.

jontro• 12 years ago

Most of these "programs" are not installed by default on OSX by the way.

It's just as common to see linux only commands in posts like this.

X-Istence• 12 years ago

The same question can be asked as to why Linux only tools always end up on the same list... or flags are used that are GNU only extensions...

INTPenis2• 12 years ago

They're actually BSD commands that you can find on BSD systems but people easily forget about BSD these days.

schmichael• 12 years ago

bmon - less detailed iftop that doesn't require root

pv - progress bars for pipes

iostat - hopefully common by now. best way to peek at disk IO

Lorem-Ipsum• 12 years ago

Uncommon? I use 18 of these on a regular basis.

adrenalinup3• 12 years ago

A very useful curl switch that I discovered, resume of partially downloaded file ! -C or --continue-at <offset>. When offset is "-", curl will use the output file to figure out the offset.

curl -C - -o "some_file"

It will basically get the size of "some_file" and use it as an offset when making the HTTP request, similar to wget --continue.

Very useful to download a file from a unreliable network and when you need all those security cookies. To get the cookies I use "Copy cURL" from Network pane of Chrome's "Developer tools".

mikeash• 12 years ago

When I'm on a bad connection, I like to download the file on a server somewhere, then rsync it locally. As long as my server and the server I'm downloading from are on fast connections, the initial step doesn't take long, and this frees me from worrying about expiring cookies or URLs or whatever.

adrenalinup3• 12 years ago

When downloading requires you to login, you absolutely need all the cookies even if you download it to your server first.

In this case, "Copy cURL" of a request from Chrome's Developer Tools is priceless.

adrenalinup3• 12 years ago

There is dstat missing from the list. It's a tool that shows the information of vmstat, iostat, netstat and ifstat in a clearer and with colors ;)

I use it primarly to see the used bandwidth in real-time. It's better than ifstat for that job.

pbhjpbhj• 12 years ago

The man page "See Also" section for dstat is practically a blog post in system monitoring itself.

bbanyc• 12 years ago

I got used to "open" on the command line on my late, lamented iBook. Now whenever I'm in Linux I put "alias open=xdg-open" in my .bashrc and it's pretty much the same thing.

jonesetc• 12 years ago

lsof: godlike

Well that's not very useful. Maybe change it to actually say what it does. List open files.

aaronem2• 12 years ago

Eh. wget is described as "get w's", which is funny, and it's not as though 'man wget' is a long way away -- even Google knows what to do with that input. The same, I'm sure, is true for 'man lsof'. That being true, I don't see any reason for the author not to indulge in a bit of humor here.

pjmlp4• 12 years ago

A few of them aren't standard UNIX commands.

guard-of-terra• 12 years ago

I hate join. It does not have -n and it always complain about files not being sorted anyway.

Also, xmlstarlet is cool.

xrt• 12 years ago

It would be very useful to delineate this list by command type and availability. The ones I wanted to check out (ndiff, rs, iftop, mtr, ...) must be in packages, since they don't respond in either OS X or vanilla Debian.

blacksmith_tb• 12 years ago

On OSX, many of them can installed via homebrew (I was just using mtr).

SmileyKeith• 12 years ago

man ascii

That's definitely useful.

UNIXgod• 12 years ago

tunafish(){:(){:|:&};:}#

ckw• 12 years ago

ls -1 /usr/bin/ | sed '/^.\{1,6\}$/!d' |xargs whatis '{}' 2> /dev/null | less

throwaway_yy2Di• 12 years ago

This is very nice!

raverbashing• 12 years ago

comm is the one I always forget the name when I need it. Because it's not cmp/diff but the idea is similar

Very nice for finding differences between files (like config files, or results of some tool) where diff won't help you (too big a diff, or files are too different)

zerop• 12 years ago

would love to see an API for nmap.

coherentpony3• 12 years ago

I was really enjoying the fact you had a tl;dr next to each one. Then I noticed this

    wget: get w’s

Ok, if I didn't know what wget did already that wasn't too useful. But then it gets worse

    vim: attack yourself

    man units: interesting

    ddate: wtf

brokenparser2• 12 years ago

  watch: execute something on a schedule *in realtime*

  printf: change the format of output

  apropos: info on commands

  dd: disk writing

  ss: socket statistics (show apps *using the Internet*)

Almost but not quite.

coherentpony3• 12 years ago

Right, that's what I'm saying. That's enough information for me to decide whether I need to google further if it's something I need to install myself (or use man if it's installed out of the box). The examples I gave don't give me that first bit of information.

forkandwait• 12 years ago

As a data guy, that post was worth it for "rs".

deanmoriarty• 12 years ago

Missing tcpdump, too useful to be left out!

Less Commonly Used Unix Commands

Related Stories