A lot of thanks to all who sent their replies.
Especially to Gavin Kreuiter, Gabor Zelenak, Dan Harrington, Jason Neil,
Joerg Bruehe, Dennis Peacock.
The problem is due to buffering.
-----Original Message-----
A bit off topic, but I wonder if someone can give me a hint why the
following doesn't work:
tail -f some.log | grep "keyline" | tee my.log
tail -f some.log | grep "keyline" works fine.
-------------------------------------------------------------
I want to filter some.log file for "keyline", store filtered output in
the file my.log
and same time watch filtered output on the screen.
I am using 4.0F and I have tried it in csh and ksh. Behaviour is same.
When I run
tail -f some.log | grep "keyline" | tee my.log
I don't see anything on my screen and nothing is going into my.log.
If I would use "cat" or even "tail" instead of "tail -f", this line
would work,
but I want to see and filter some.log continuously.
Though, it's hard to see, what is the culprit.
Because
tail -f some.log | grep "keyline" > my.log
behaves in the same way. There are no input into my.log for 5 min and
more.
tail -f some.log | tee my.log
works OK.
I see filtered output immidiately and my.log is being filled in
immediately.
But the construction is useless for me because there is no filtering
done.
As well
tail -f some.log | tee my.log | grep "keyline"
starts to output immediately on screen and to my.log
but my.log is still unfiltered and it's not what I want.
So the problem still stays:
How to filter out some continuously growing log file,
save filtered output into a file and watch it on the screen same time
without delays.
Here below some of the responses:
Just as a guess, I'd say you're probably seeing the effect of buffering
in
the tee utility...because 'tail -f' is an ongoing process, the pipeline
is
staying open, and tee is still running, accepting input. If my.log
isn't
being updated, it's probably due to tee not having written out it's
results
until it gets an EOF (or some good size chunk of data). Or perhaps it's
writing to a temp file, and only updates the final output file once it's
finished.
-----------------------------------------------------
I don't know *exactly* why this is but believe it's because the the
stdout
from the grep command never gets to the tee command. Possibly something
to
do with the tail never terminating, hence the grep never terminates and
perhaps that stops the std out from being passed to the next pipe. If
you
open another window and kill the tail command you'll see your pipeline
splurt out everything it grep'ed to date.
A work around would be to redirect your grep'ed tail to a file, in the
background perhaps, and tail -f the file you are redirecting to. messy
i
know.
-----------------------------------------
If you send output to a file or pipe, it is by default block-buffered
in the "stdio" functions. So your command IMNSHO will still "work",
but the version including 'tee' does have a much larger delay.
-----------------------------------------
I am on Tru64 version 5.1 PK3 and if I cat a file...say the
/var/adm/messages file and pipe that to grep some_text and pipe that to
tee
my.log it works fine. My suspicion is the trouble lies with the tail -f
output to grep and then out to the tee.
----------------------
"At a guess I would say it is a buffer flush problem. I am sure that if
you
waited long enough, data would finally appear in my.log. The -f never
allows tail to complete, so tee never writes its log; breaking out of
the
tail simply breaks the tee without flushing tee's buffer.
tail -f some.log | grep "keyline" >my.log
should work, but of course you won't see the terminal output.
I found that it IS a buffer problem; the file is not
written to until 8K of buffer is filled. If your command generates more
than 8K, you will (eventually) see the first 8K written.
--------------------------------------------------------------
From: Tarasyuk Nik
Sent: Thursday, October 18, 2001 4:09 PM
Nik Tarasyuk
Snowy Hydro
Australia
Received on Fri Oct 19 2001 - 05:02:55 NZDT