On Aug 2, 2007, at 9:57 PM, Bill Briggs wrote: > At 9:06 PM -0400 8/2/07, David Livesay wrote: >> Every example I've seen looks basically like this: >> >> for i in `cat somefile` >> do >> echo $i >> done >> >> But this only works if the lines have no whitespace in them, which >> doesn't exactly describe most of the files I'm trying to work with >> (log files, to be specific), because i ends up being each word >> instead of each line. >> >> I could pipe the output from cat to sed and replace the whitespace >> with a character not found in log files and then replace those >> characters with whitespace before processing them, but that seems >> like a waste of time to me. (Actually, that pretty much sums up my >> impression of shell scripting so far, but I have something I need >> to accomplish with it, unfortunately.) > > Before answering this it would be nice to know what your goal was. > I haven't a clue from the above. What is it you have for starting > material (some log file I presume?) and what are you trying to do > with it, or get from it. Your shell script just doesn't seem to do > much that I can see as being useful. Okay, my goal is to iterate through a log file that hasn't been archived on a regular basis as it should have been and create monthly archives, leaving only the current month in the current log. In Tiger, secure.log gets archived weekly (or whenever the 500.weekly script runs, which I haven't totally figured out yet) which is a little excessive, but Panther doesn't archive them at all. This creates a problem for generating monthly usage statistics on public computers. If you grep all the entries that start with "Jun" you might get three years' worth. So I've written a log roller for monthly.local, which solves the problem going forward, but I'd like to add a log archiving utility to the install script that installs the monthly.local file. The simplest way to do this would be to iterate through secure.log line-by-line and write each line to a file until the name of the month changes, then create a new file and gzip the previous one. That way, the entries from bygone years will get written to different files, and ultimately deleted, probably, but we haven't decided yet. > Don't be so quick to disparage the shell. UNIX is the underbelly > of the internet we all know and love (or not). It's very capable > and very fast. And don't get discouraged if you can't seem to > contain it all in your head. It's a cognitive black hole. You can > spend years on UNIX and you'll still have as much to learn as you > did at the start. But it can still give you great power. Yes, I've noticed how powerful Unix geeks are. :-) >> BTW, can anyone recommend a good book on shell scripting for >> someone who doesn't intend to make a career of it--just get a few >> things done? I've done about all I can with man pages. > > You could buy a book, but if you're on OS X 10.4.x you'll not do a > lot better than this free on-line source. It covers advanced bash > scripting, and bash is the default shell in 10.4.x (unless you > changed it). If you started out with OS X beta you got tcsh as your default shell. If you kept all your files and settings every time you upgraded your OS imported them every time you upgraded your hardware, then tcsh would still be your default shell. That's what I did. I'm pretty comfortable with it now, and I like it, but I usually write my scripts for sh so it will run anywhere. If I ever run into something where I need a bash or tcsh feature in a script, I'll use bash or tcsh, but I haven't yet. > http://www.tldp.org/LDP/abs/html/ This is pretty good. He includes sh as well as bash and points out the differences. This guy should write a book. He even explains how to iterate a file line-by-line: cat secure.log | while read line; do echo $line done By the way, this script doesn't do much either, but it does tell you what $line contains. When it contains what you want it to contain then you can go on and write something that does something useful, like this: mnth=null cat secure.log | while read line; do m=`echo $line | cut -c1-3` if [ $mnth = $m -o $mnth = null ]; then if [ $mnth = null ]; then mnth=$m; fi # capture month from current line # write each line to secure.log.0 until the month changes echo $line >> secure.log.0 else mnth=$m # if there is already an archived file, rotate it and any other archives if [ -f secure.log.0.gz ]; then for i in `ls -r secure.log.*`; do if [ -f "${i}" ]; then num=`echo $i | sed -E 's/secure\.log\.([0-9]+)\.gz/\1/'` num=`expr $num + 1` mv -nv "${i}" "secure.log.${num}.gz" fi done fi # archive the current secure.log.0 gzip -v9 secure.log.0 fi done if [ -f secure.log.0 ]; then # this is the current month, so write it back to secure.log mv -f secure.log.0 secure.log fi > There's a downloadable PDF version of it too. If you do use this, > one caveat. He says that you can use the extension .sh for shell > scripts. No can do. You'll need to use .bash on your Mac. Even if > you specify the path to the bash shell in the first line of the > file it won't use bash without the file extension. This will cause > some examples to fail. So just keep that file extension in mind as > you go and it'll work just fine. I generally omit the extension unless it's mandated, like monthly.local or weekly.local, for example. Most of the scripts in / usr/bin don't have extensions. > - web > > P.S. It helps to read man pages if you've had some exposure to C > programming. If not, then some portions of some man pages will seem > exceedingly impenetrable. I've gradually gotten to where I can interpret most man pages, but some entries are just plain useless (see man read, for example) and most of the really important things you need to write a shell script aren't documented. There's more to writing a shell script than knowing a few commands. man for, for example, tells you all about how for loops work--in Tcl. Anyway, thanks for the reference, Bill. My script is done; I'll see how it works tomorrow.