First message out here on the group, and I hope you can help me out.
I am running a bunch of commands that can take a long time (10 seconds to 15 minutes and beyond). Naturally, I decided to multithread them, which makes them run a lot faster since I have a multi-core machine. The problem is they all access a log file in the current directory, and they collide.
So, my next step was to use Dir.chdir to move to another directory for each thread and run there. Now it works.
Now, my problem is that I have been using backticks (`) to run commands, and I would like to switch to popen in order to stifle the output (it spits out a lot of info that I don’t want all over my tests, and it generates odd characters that mess up my terminal).
The problem is that popen doesn’t respect Dir.chdir. I have also tried things like:
popen(“cd otherdir; command”)
and
popen(“cd otherdir && command”)
but it doesn’t seem to care at all, and always runs in rails root.
So, my next step was to use Dir.chdir to move to another directory for each
thread and run there. Now it works.
This is kind of hackish. Are you sure the binary does not have a
switch or option to change where the log file goes? Almost all the
standard Unix utils that have logfiles let you modify this. If you
have access to the source, adding such a flag is trivial. If the flag
exists, simply appending a "random number" (I use the unix timestamp
plus another counter (just in case I start two thread in the same
second) myself quite often) to a common log name will usually
suffice. This is also handy because you can then pull the date and
time it was executed from the log file's name, if you don't need a
running log with detailed timestamps.
So I need one of two things:
1) Change directory for popen
2) Stifle stdout and stderr for `command`
2 is easy. The backtick style of calling commands actually spawns a
shell, so shell-based redirects to /dev/null (as well as file
globbing, scripting, conditional execution, etc.) work as if you were
actually at your command line.
So, my next step was to use Dir.chdir to move to another directory for each
thread and run there. Now it works.
This is kind of hackish. Are you sure the binary does not have a
switch or option to change where the log file goes? Almost all the
standard Unix utils that have logfiles let you modify this. If you
have access to the source, adding such a flag is trivial. If the flag
exists, simply appending a "random number" (I use the unix timestamp
plus another counter (just in case I start two thread in the same
second) myself quite often) to a common log name will usually
suffice. This is also handy because you can then pull the date and
time it was executed from the log file's name, if you don't need a
running log with detailed timestamps.
I know it's hackish
The reason I can't use a switch for the log file is that the program I
am calling (which does have a log file switch) calls another program
(which has a log file switch). But, the first program does not have a
log file switch for the second program's log file.
So I need one of two things:
1) Change directory for popen
2) Stifle stdout and stderr for `command`
2 is easy. The backtick style of calling commands actually spawns a
shell, so shell-based redirects to /dev/null (as well as file
globbing, scripting, conditional execution, etc.) work as if you were
actually at your command line.
I tried piping the output using >, but I didn't realize my program was
actually outputting to stderr, so I just changed it to be 2> /dev/null
and now its nice and quiet.
