Pipe Viewer

Right. I usually use pv in place of 'cat' at the start of a long set of piped commands. Anywhere you could use 'cat' you can use pv. If I'm not mistaken, I think you can also use multiple 'pv' commands at a time too.

lloeki• 14 years ago

This is indeed visible in the example, only source has progress while bzcat and gzip only have flow.

bsg752• 14 years ago

You have to supply it the stream sizes if you want completion percentages, otherwise it displays rates only.

Example: https://gist.github.com/2783910

dbuthay• 14 years ago

It's amazingly useful to have realtime throughput rates

tail -f query.log | pv -tri 1 > /dev/null

pramodbiligiri• 14 years ago

Nifty command. Been using this for a month or so. One use case is measuring if a downstream process can consume data as fast the previous process is producing it (by alternating second process with /dev/null or some such).

spudlyo• 14 years ago

It's also great that pv supports rate limiting, so you can choose if you want to saturate your network when you use it with netcat.

there2• 14 years ago

Somewhat related, trickle is a good user-space bandwidth shaper if you aren't sending data over a pipe: http://monkey.org/~marius/pages/?page=trickle

ta12121• 14 years ago

http://news.ycombinator.com/item?id=3198572

http://news.ycombinator.com/item?id=2567186

gouranga• 14 years ago

Nice - that'll spice up my rather boring svnadmin dump sessions - thanks for posting!

ralph• 14 years ago

Note, a plain dd will report progress on stderr if sent a SIGUSR1. That can sometimes be enough to avoid the overhead of pv(1) and sending huge amounts of data through another pipe.

protopete• 14 years ago

About the pv overhead, I sent a note to the pv author saying that on Linux at least the transfer function could take advantage of the splice system call. Splice "copies" data from one file descriptor to another file descriptor, avoiding having to copy the data from the kernel to userspace and back.

Erwin• 14 years ago

On FreeBSD (and maybe Mac OS X) you can press Control-T to send a SIGINFO signal (non-standard extension. This prints out how far dd has gotten (and perhaps other FreeBSD utilities too).

there2• 14 years ago

You may need to setup your terminal to do that with:

    stty status '^T'

fsck also responds to SIGINFO by printing its status. I had to hack up rsync to do it (http://jcs.org/patches/rsync-show_current_file.diff), and I wish more utilities supported it by default.

sneak• 14 years ago

N.B. also that dd on osx will EXIT IMMEDIATELY if sent a SIGUSR1. It double-sucks because one tends to do this on long-running dd processes that you're wondering are done yet...

kilovoltaire2• 14 years ago

yes | pv > /dev/null

sp332• 14 years ago

  pv /dev/zero > /dev/null

kilovoltaire2• 14 years ago

awesome, i was actually wondering if there was a "yes-esque" (yesque?) /dev/ file.

my throughput from yes is 49MB/s whereas from /dev/zero it's 12GB/s

potatohead00• 14 years ago

Found out about pv ~year ago when working with large log files. Very useful. I also like it for testing how quickly a script can consume data.

b0b0b0b• 14 years ago

If I'm dealing with [compressed] files on disk or across a network, what's the practical overhead of pv? A few extra percent CPU, or some material decrease in through put?

blibble• 14 years ago

very very little, it's just doing read(), counting the bytes, then write(), with some screen output every few seconds.

(hell it could even use splice() and never even see the bytes)

Pipe Viewer

Related Stories