On Aug 2, 2007, at 9:57 PM, Bill Briggs wrote:
> At 9:06 PM -0400 8/2/07, David Livesay wrote:
>> Every example I've seen looks basically like this:
>>
>> for i in `cat somefile`
>> do
>> echo $i
>> done
>>
>> But this only works if the lines have no whitespace in them, which
>> doesn't exactly describe most of the files I'm trying to work with
>> (log files, to be specific), because i ends up being each word
>> instead of each line.
>>
>> I could pipe the output from cat to sed and replace the whitespace
>> with a character not found in log files and then replace those
>> characters with whitespace before processing them, but that seems
>> like a waste of time to me. (Actually, that pretty much sums up my
>> impression of shell scripting so far, but I have something I need
>> to accomplish with it, unfortunately.)
>
> Before answering this it would be nice to know what your goal was.
> I haven't a clue from the above. What is it you have for starting
> material (some log file I presume?) and what are you trying to do
> with it, or get from it. Your shell script just doesn't seem to do
> much that I can see as being useful.
Okay, my goal is to iterate through a log file that hasn't been
archived on a regular basis as it should have been and create monthly
archives, leaving only the current month in the current log.
In Tiger, secure.log gets archived weekly (or whenever the 500.weekly
script runs, which I haven't totally figured out yet) which is a
little excessive, but Panther doesn't archive them at all. This
creates a problem for generating monthly usage statistics on public
computers. If you grep all the entries that start with "Jun" you
might get three years' worth. So I've written a log roller for
monthly.local, which solves the problem going forward, but I'd like
to add a log archiving utility to the install script that installs
the monthly.local file. The simplest way to do this would be to
iterate through secure.log line-by-line and write each line to a file
until the name of the month changes, then create a new file and gzip
the previous one. That way, the entries from bygone years will get
written to different files, and ultimately deleted, probably, but we
haven't decided yet.
> Don't be so quick to disparage the shell. UNIX is the underbelly
> of the internet we all know and love (or not). It's very capable
> and very fast. And don't get discouraged if you can't seem to
> contain it all in your head. It's a cognitive black hole. You can
> spend years on UNIX and you'll still have as much to learn as you
> did at the start. But it can still give you great power.
Yes, I've noticed how powerful Unix geeks are. :-)
>> BTW, can anyone recommend a good book on shell scripting for
>> someone who doesn't intend to make a career of it--just get a few
>> things done? I've done about all I can with man pages.
>
> You could buy a book, but if you're on OS X 10.4.x you'll not do a
> lot better than this free on-line source. It covers advanced bash
> scripting, and bash is the default shell in 10.4.x (unless you
> changed it).
If you started out with OS X beta you got tcsh as your default shell.
If you kept all your files and settings every time you upgraded your
OS imported them every time you upgraded your hardware, then tcsh
would still be your default shell. That's what I did. I'm pretty
comfortable with it now, and I like it, but I usually write my
scripts for sh so it will run anywhere. If I ever run into something
where I need a bash or tcsh feature in a script, I'll use bash or
tcsh, but I haven't yet.
> http://www.tldp.org/LDP/abs/html/
This is pretty good. He includes sh as well as bash and points out
the differences. This guy should write a book.
He even explains how to iterate a file line-by-line:
cat secure.log |
while read line; do
echo $line
done
By the way, this script doesn't do much either, but it does tell you
what $line contains. When it contains what you want it to contain
then you can go on and write something that does something useful,
like this:
mnth=null
cat secure.log |
while read line; do
m=`echo $line | cut -c1-3`
if [ $mnth = $m -o $mnth = null ]; then
if [ $mnth = null ]; then mnth=$m; fi # capture month from current
line
# write each line to secure.log.0 until the month changes
echo $line >> secure.log.0
else
mnth=$m
# if there is already an archived file, rotate it and any other
archives
if [ -f secure.log.0.gz ]; then
for i in `ls -r secure.log.*`; do
if [ -f "${i}" ]; then
num=`echo $i | sed -E 's/secure\.log\.([0-9]+)\.gz/\1/'`
num=`expr $num + 1`
mv -nv "${i}" "secure.log.${num}.gz"
fi
done
fi
# archive the current secure.log.0
gzip -v9 secure.log.0
fi
done
if [ -f secure.log.0 ]; then
# this is the current month, so write it back to secure.log
mv -f secure.log.0 secure.log
fi
> There's a downloadable PDF version of it too. If you do use this,
> one caveat. He says that you can use the extension .sh for shell
> scripts. No can do. You'll need to use .bash on your Mac. Even if
> you specify the path to the bash shell in the first line of the
> file it won't use bash without the file extension. This will cause
> some examples to fail. So just keep that file extension in mind as
> you go and it'll work just fine.
I generally omit the extension unless it's mandated, like
monthly.local or weekly.local, for example. Most of the scripts in /
usr/bin don't have extensions.
> - web
>
> P.S. It helps to read man pages if you've had some exposure to C
> programming. If not, then some portions of some man pages will seem
> exceedingly impenetrable.
I've gradually gotten to where I can interpret most man pages, but
some entries are just plain useless (see man read, for example) and
most of the really important things you need to write a shell script
aren't documented. There's more to writing a shell script than
knowing a few commands. man for, for example, tells you all about how
for loops work--in Tcl.
Anyway, thanks for the reference, Bill. My script is done; I'll see
how it works tomorrow.
|
