Make your STDERR red

https://github.com/sickill/stderred

by ku1ik • 15 years ago

133 88 15 years ago

github.com

joh6nn2• 15 years ago

thank you. this is probably the cleanest way to do this i've seen, and i've seen a LOT of ways to do this.

anon_d• 15 years ago

How the hell is this even remotely clean?

burgerbrain16• 15 years ago

Yeah, this is a tad hackish for my tastes honestly, makes to many assumptions.

The way I came up with while brainstorming a while ago (never got around to fully implementing it) is to create a PTY wrapper that you can fire off a program with.

Basically it creates a new PTY and wires it up to it's own controlling PTY, and forks of a child under the new one. You can then trivially do a lot of things transparently, like separating stdout and stderr (normally both stdout and stderr would be attached to the slave side of the PTY (/dev/tty), but if you attach them to a different fd then the select on the master/parent side can do things like color them).

EDIT: if it helps you picture it, this is basically doing userland STREAMS ;)

I'd estimate you could do this for around 250 lines of C. Maybe I'll see what I can do after dinner.

joh6nn2• 15 years ago

"cleanest way i've seen" != "ideal"

it isn't a horrible mangling of pipes and/or file descriptors, or some custom wrapper script that you have to run before every command, or any of the other frightening ways to do this that i've seen. it's a simple library that you can load or unload easily, that works system wide, without being an impenetrable, unmaintainable mess. as far as these things go, that's clean.

J_Darnley2• 15 years ago

Er, why would you want all the text to be red?

burgerbrain16• 15 years ago

Not all of it, just stderr (so not stdout). This is useful because many things use stdout for regular output and stderr for error output (some things just use stdout for both, but that's not very nice as it disables doing things like this).

bouncingsoul2• 15 years ago

I think they're suggesting just styling "Error:" in red and then using normal text for the actual message.

npm does something like that: http://i.imgur.com/7oBLP.png

wladimir2• 15 years ago

Well if you don't like red, with a 256-color terminal emulator you could change it into a nice shade of purple instead:

    static const char PURPLE[]     = "\x1b[38;5;129m"
    #define STDERR_COLOR PURPLE

Or something more sane: http://www.mudpedia.org/wiki/Xterm_256_colors

J_Darnley2• 15 years ago

To reply to everyone here, I still don't get why you want all text on stderr to be coloured, be it red or purple.

Most programs can write data to stdout so they put all messages on stderr. Making it all coloured hides the errors in the rest of the text.

Making errors red, or any other colour, is a good idea. It makes "under-experienced users" notice them.

burgerbrain16• 15 years ago

I'm getting terribly confused by your confusion, so forgive me if I'm not addressing your concern..

If the program is mixing "info" level messages and "error" level messages (to use the syslog terms for them) on stderr, there really isn't much we can do about it. The understanding however is that things on stderr are probably things that the user wants to read.

Coloring both of these red shouldn't hide them anymore than coloring both of them white, except now they are both easily discernible from the 'data' coming out of stdout (which is our goal).

Possible aside: the proper way to write/read from the terminal in cases where your stdin/stdout were redirected is to just open up /dev/tty yourself. Any program that does that will not have that text colored by any of the solutions I've seen here.

Xuzz• 15 years ago

On OSX, you might be able to do something similar using dyld's interposing feature (http://opensource.apple.com/source/dyld/dyld-132.13/include/...) to replace write() and the dyld equivalent to LD_PRELOAD, DYLD_INSERT_LIBRARIES.

wmobit• 15 years ago

The alloca usage here terrifies me.

burgerbrain16• 15 years ago

That is a good point. I don't know if write has a limit on 'count' (I can't find anything saying that it does), but if it's simply "whatever you can fit in size_t" then you certainly want to be careful about putting that on your stack. Failed alloca's don't really give you any indication, though you might sigsegv later.

sickill13• 15 years ago

What would you suggest instead then? I'm not C programmer.

tlrobinson2• 15 years ago

Nice hack. Ideally it would just be a feature of the shell. At least it does a isatty check though.

Is there a reason you allocate a bunch of memory and copy the strings rather than just doing multiple writes?

    (*lol_write)(fd, STDERR_COLOR, STDERR_COLOR_SIZE);
    (*lol_write)(fd, buf, count);
    (*lol_write)(fd, COL_RESET, COL_RESET_SIZE);

bdonlan3• 15 years ago

Or use writev for that matter.

Note that using ordinary write multiple times has atomicity implications - if you have multiple processes writing to STDERR, you might find their output interleaved in unfortunate ways. Using a single write with a temporary buffer or writev avoids this (provided you're not writing more than PIPE_BUF bytes total)

nviennot2• 15 years ago

Indeed: https://github.com/sickill/stderred/pull/1

kelnos2• 15 years ago

The reason would be interleaving if more than one process is writing to stderr.

I was more bothered by the use of alloca(). Even on a 32-bit system with a 32-bit size_t, you can blow your stack to hell with a single call to this modified write(). Having to do a full malloc() here would be pretty lame, but as others suggested, writev() should do nicely.

The other nice property of using writev() is that you don't have to bother with the dlopen/dlsym crap. Can just call writev() directly from the overridden write() in either case.

Of course, that doesn't catch people writing to stderr using writev() directly, but I think that's ok. (The current impl doesn't catch that case anyway.)

(edit: pull request submitted! https://github.com/sickill/stderred/pull/6)

sickill13• 15 years ago

Like bdolan said, it would have atomicity implications this way. I agree it would be lighter on memory but in some cases output might break if many processes were writing to stderr at the same time.

derleth3• 15 years ago

Ubuntu does indeed allow multi-arch distros. I've run 32-bit Firefox on 64-bit Ubuntu for years; in fact, I'm doing so now.

sickill13• 15 years ago

Allright, maybe. But it still has DST ($LIB token) disabled in LD_PRELOAD.

derleth3• 15 years ago

Maybe? What's maybe about it? I can build (using -m32) and run 32-bit x86 binaries on an x86-64 platform.

tlrobinson2• 15 years ago

On OS X:

    gcc stderred.c -D_GNU_SOURCE -Wall -ldl -fPIC -shared -o lib/stderred.dyld
    alias stderred='DYLD_INSERT_LIBRARIES=$PWD/lib/stderred.dyld DYLD_FORCE_FLAT_NAMESPACE=1'
    stderred python -c 'import os; print "Yo!"; os.write(2, "Jola\n\r")'

(replace $PWD with the absolute path if you're adding this to your shell config, of course)

sickill13• 15 years ago

Cool, looks like the same gcc flags works for OSX.

raldi• 15 years ago

Why override write() when you can just use pipes?

http://distfiles.exherbo.org/distfiles/hilite-1.5.c

vilhelm_s2• 15 years ago

One problem is stdio buffering. By default the C stdio functions will flush buffers after every line for file descriptors that are connected to terminals, but only every now and then for file descriptors connected to disk files or pipes. So if a program outputs messages on both stdout and stderr (like a compiler), then after filtering one of the streams through a pipe, the messages are not interleaved properly.

colmmacc2• 15 years ago

One way is to wrap the program with a parent process that can handle the colourising. Here's a version I've been using for 6 years or so;

http://people.apache.org/~colm/utils/sexec.c

it uses regular select semantics and a trivial state machine to pick the active fd, and has an XML mode - which is convenient for colourising in XHTML (if you want to record an automated process say).

udp• 15 years ago

I may be wrong, but I always thought stderr was for diagnostic messages as well as errors, in which case red wouldn't really be appropriate.

I almost think we need another output stream for something that's neither an error nor output of the program.

jzwinck• 15 years ago

C++ has std::clog (http://www.cplusplus.com/reference/iostream/clog/) but most people haven't heard of it and it maps to stderr anyway.

Jach• 15 years ago

I don't know, I kind of have mixed feelings about adding another stream. I'd probably find stdinfo and stdwarn useful. But it's a slippery slope to duplicating the features of a full logging system; you might add stdinfo, stdwarn, stdfine, stdfiner, stdfinest, stdconfig, std$custom, etc. etc.

One nice thing about having only stdout and stderr is it pushes you to ask: "Is this piece of information actually necessary to display to the user or should it just be logged to a file somewhere?" I'm not sure everyone really asks that though. Every time someone shoves a bunch of needless information to stderr "just in case" is a time I have to 2> /dev/null. (At least the useless stuff is often in stderr so I don't have to grep it out of stdout.)

anko2• 15 years ago

I think this is a marvellous idea.

I would love to see a meta stream, that allows you to mark up semantics of stdout/stderr.

How would you go about creating a new standard stream?

strlen• 15 years ago

Why did you truncate a size_t to an int?

sickill13• 15 years ago

Will fix that, thx.

apinstein2• 15 years ago

Or, add this to .zshrc:

# colorize all stderr red exec 2>>(while read line; do print '\e[91m'${(q)line}'\e[0m' > /dev/tty; print -n $'\0'; done &)

works like a charm.

1amzave• 15 years ago

Though I'm not a zsh user, my first reaction was likewise "meh, my shell can do that", and so I whipped up a pretty much identical command (only using sed). However, when it didn't behave quite as I expected, I dug a little deeper and ended up with (what was to me) a curious discovery: bash prints its prompt to stderr rather than stdout (see http://git.savannah.gnu.org/cgit/bash.git/tree/parse.y#n5013).

Anyone know why it does that?

burgerbrain16• 15 years ago

That's a great question. Zsh and tcsh both use stdout for their prompt, ksh and dash use stderr for the prompt itself but stdout for whatever the user is typing into the prompt. Bash, at least as it's configured on my system, seems to use stderr for the prompt and whatever the user types on it.

I really can't think of a reason that the way tcsh and zsh handle this isn't the proper way. Surely the principle of least surprise applies here.

sickill13• 15 years ago

This is the solution I started with. However it's messing line order sometimes due to race conditions/buffering.

apinstein2• 15 years ago

Yeah we're getting that issue, too. And sometimes the prompt doesn't appear properly.

tzury• 15 years ago

This is my python version (for logger)

https://gist.github.com/1474991

noeleon• 15 years ago

can it somehow incorporate the blink tag as well?

burgerbrain16• 15 years ago

(Unfortunately) yes, only some terminal emulators support it though.

bdonlan3• 15 years ago

This actually doesn't work for processes using stderr on eglibc 2.13 - fprintf internally outputs via a different path that can't be hooked in this manner. On x86, you could try overwriting the system call gate pointer to a hook function, but on x86_64 you're basically stuck, unless you attach a debugger or something.

Also, the write prototype is incorrect (using int instead of size_t) and could break on 64-bit machines.

sickill13• 15 years ago

Thx, will fix int -> size_t.

mwsherman• 15 years ago

I like command lines, but often make fun of people that prefer them over well-produced GUIs. I suppose the reason is that well-produced GUIs are rare.

That being said: turning text red is notable? In 2011?

(Will accept downvotes with dignity.)

cleverjake• 15 years ago

It's less of a unique idea, and more of a why-didn't-I-think-of-that sort of thing, in my opinion.

wladimir2• 15 years ago

What, you're trying to start a GUI versus command-line argument? On hacker news? In 2011?

bnegreve• 15 years ago

Turning all error messages red systemwide is a neat result.

sovande2• 15 years ago

The concept may be useful for someone, but the implementation is just slow, naive and definitely not something you want to interpose your libc write function with. If the author is reading this; 'man (2) writev' for a quick code and efficiency improvement.

sickill13• 15 years ago

I am not C programmer. Actually this piece of code is my first C code since over 10 years. I might have forgotten about all good practices and stuff. But it works for me so I shared. Would you like to contact me and point slow, naive points or even better improve it and send pull request?

sovande2• 15 years ago

The slow point mainly refers to allocating and copy into a new buffer. The naive points refers to that you use 1) alloca to allocate an unknown amount of memory on the stack and 2) assume that write is atomic, never fails and will write the whole buffer (you return count). Instead you should return what write actually wrote or not. There are also a whole can of worms with regards to what happens during (signal) interrupt or if write failed to write the whole buffer. In which case the reset color code may not be sent etc etc. Solutions to this require some fine-thinking, but for the slow stuff, you can simply replace all the allocation and copy with something like this

  struct iovec iov[3] = {{STDERR_COLOR, STDERR_COLOR_SIZE},{buf, count},{COL_RESET, COL_RESET_SIZE}};
  ssize_t n = 0;
  do { n = writev(2, iov, 3);} while (n == -1 && errno == EINTR);
  return n;

cnvogel• 15 years ago

While it might not be relevant in practice, there's the possibility of data corruption if someone closes all filedescriptors (for whatever reason) and then operates on the newly generated ones. This implementation just checks for filedescriptor number 2, a more sensible implementation for sure should hook the stdio FILE or iostream mechanisms to restrict colorization only to the "standard error" case.

for(i=0;i<1024;i++) close(i); /* close a lot of fd /

/ open will yield the lowest free fds, here it's 0,1,2 /

f=open("inputfile",O_RDONLY);

g=open("outputfile1",O_RDWR|O_CREAT,0666);

h=open("outputfile2",O_RDWR|O_CREAT,0666);

read(f,buf,1024);

write(g,buf,1024);

write(h,buf,1024); / <-- data is colorized! */

kelnos2• 15 years ago

The isatty() check should cover most cases of badness. Assuming "outputfile2" is a regular file, the data will not get colorized in your example.

sickill13• 15 years ago

Thx for catching this. Interesting in making pull request? ;)

sickill13• 15 years ago

Btw, as kelnos said, isatty() check "should" prevent all bad scenarios.

jrabone6• 15 years ago

Well, I suppose it's one step up from embedding ANSI escape sequences in your output. Various crappy Ruby tools do this and it makes Windows development SUCH a joy.

Free clue: the world is not a 1960s UNIX terminal. If you don't want to support Windows, that's fine - don't. I'll use something else or write my own. If you DO claim to "work" on Windows, read the goddamn docs and don't just blindly emit escape sequences on std(out|err) because it DOES NOT WORK.

It's not that hard to support Windows console IO, and it's well documented. I've previously written a (commercial) Java native library to provide curses-style character-at-a-time support for Java running in a terminal, using curses on UNIXalikes and a choice of Win32 console or Cygwin curses on Windows, although Cygwin curses at the time didn't correctly recognise Ctrl-Space (makes supporting Emacs keybindings a non-starter). Win32 has a considerably saner console implementation IMO.

alextingle• 15 years ago

What a bizarre rant.

This is a libc/LD_PRELOAD hack, so it's clearly a Unix-only solution. Where did all these "you'll break my Windows console" tears come from??

jrabone6• 15 years ago

What's bizarre about it? Clearly this library isn't the problem. What'd be even better would be a sane console standard that isn't stuck with being backwards compatible with a 7 bit everything-is-a-character protocol. The acoustic coupler is dead people, get over it! Relegate termcap / terminfo / curses / getty to the same bin we dumped Gandalf serial line multiplexors in, and move forward!

roel_v2• 15 years ago

It's quite a pain though to get console coloring to work properly on Windows. I wrote a utility once that batch scripts could call to set the fore/background color (just a call to SetConsoleTextAttribute() really) but in practice it's quite a pain to write scripts that color their output properly (especially once you start mixing languages of those scripts). The terminal-emulator-that-interprets-escape-sequences-as-color-change-commands is much easier for that purpose.

xutopia• 15 years ago

Are you forced by work to use Windows?

jrabone6• 15 years ago

Actually, no - I use Linux on the desktop at work, and Windows by choice elsewhere. However, after more than 10 years of pain I've had it with cross-platform code that isn't. I KNOW that this library isn't SUPPOSED to be cross-platform, that was my point - at least this is better than hardcoded ANSI escapes in strings!

anko2• 15 years ago

with the stderred LD_PRELOAD alias this works for me: stderred python -c 'import os; print "Yo!"; os.write(2, "Jola\n\r")' but this doesn't; stderred ruby -e 'puts "Yo!";STDERR.write "Jola\n\r"'

if i do an strace the writes are going to the same file descriptors.. any ideas?

sickill13• 15 years ago

Maybe ruby's write() is not using write() from libc but other function like fprintf?

Make your STDERR red

Related Stories