Distributed grep client in Go

by irtefa • 13 years ago

23 15 13 years ago

microtonal2• 13 years ago

This doesn't really particularly safe to me (assuming that exec.Command allows spaces in arguments):

  cmd := exec.Command("grep", s, logName)

Since you are searching for the expression in just one log file, why not use Go's excellent regexp package?

irtefa2• 13 years ago

It started off as a class assignment where it was required to execute "grep" on whatever shell we were running the program on. That's why, instead of using the regexp package, we went for the above (somewhat shitty) approach. But, that's something I am definitely looking into changing!

enneff• 13 years ago

You might have to work pretty hard to write a grep that's as good as grep. It pulls a lot of swift moves in the name of speed.

cmccabe2• 13 years ago

exec.Command doesn't run a shell; it just executes the command. So yes, it's perfectly safe to have spaces and other shell metacharacters in there. Whether or not it's efficient here is another question, of course.

microtonal2• 13 years ago

So yes, it's perfectly safe to have spaces and other shell metacharacters in there.

You could (for instance) probe content of system files by passing

  -f/some/file

And matching that to the log. If you have some control over the log, you could use this to read content from system files. Of course, it's all indirect, since the file is interpreted as containing regular expressions. But it's certainly not 'perfectly safe'.

easytiger3• 13 years ago

I don't know if this is meant to be actually used, but clearly ssh with shared keys and a trivial bash script would be better than just shelling out to grep from go itself. Fairly pointless really.

tsenart• 13 years ago

Without the context of why this was developed, I have to agree with easytiger's point of pointlessness. Perhaps it was just an exercise in curiosity and exploration of the language. If not, I would use pdsh instead http://sourceforge.net/projects/pdsh/ $ pdsh -b -w '10.0.0.1,...,10.0.0.n' 'cat /var/log/your_log | grep "your_pattern"'

easytiger3• 13 years ago

Yea i dont mind things like this being posted, but some context would be good. It seems to present itself like a serious tool which might dupe some inexperienced people.

maguireb• 13 years ago

Nice one on pdsh! Hadn't heard of it before but will definitely be using it from now on. No more bash for loops with ssh :)

willvarfar• 13 years ago

I've just got a little python script that sshs around my boxes running grep on each, writing it to the a log file with the box name prefixed, and sorting them all by date (as my log lines have date prefixes and this makes sense for me).

Using ssh instead of running a service on each server seems more straightforward to my thinking.

mcot2• 13 years ago

This isn't really 'distributed grep' as far as I can tell. It is simply running grep individually on each machine.

satish80• 13 years ago

Golang dependency on each machine can be removed by building the executable. The command to execute the server cannot be run remotely. Since the grep is confined to a single log file, it is not very useful. It shows how easy to use go routines to make concurrent requests.

Distributed grep client in Go

Related Stories