Quick Tip: Sequences and bash expansion

September 16, 2007

It’s a lazy Sunday, and I was doing some quick clean up work and consolidation of old log files. I rotate mine by month and day, and so I end up with many log files in a directory with sequential names. Sometimes, I’ll want to do analysis on only a few of these days or a range of days — for example, it can be useful to concatenate apache log files into a single file for easy processing with awstats, Urchin, or webalizer. Here are a couple quick ways to deal with sequential files and pick what you need.

Given a directory full of sequential files:

[mshade@opteron mysql]$ ls 08*
08012007.tar.bz2 08082007.tar.bz2 08152007.tar.bz2
08222007.tar.bz2 08292007.tar.bz2 08022007.tar.bz2
08092007.tar.bz2 08162007.tar.bz2 08232007.tar.bz2
08302007.tar.bz2 08032007.tar.bz2 08102007.tar.bz2
08172007.tar.bz2 08242007.tar.bz2 08312007.tar.bz2
08042007.tar.bz2 08112007.tar.bz2 08182007.tar.bz2
08252007.tar.bz2 08052007.tar.bz2 08122007.tar.bz2
08192007.tar.bz2 08262007.tar.bz2 08062007.tar.bz2
08132007.tar.bz2 08202007.tar.bz2 08272007.tar.bz2
08072007.tar.bz2 08142007.tar.bz2 08212007.tar.bz2
08282007.tar.bz2


If I want to look at only those files from between August 10 and August 23, I can use this expression to take advantage of bash’s built in sequence expansion:

[mshade@opteron mysql]$ ls 08{10..23}*
08102007.tar.bz2 08132007.tar.bz2 08162007.tar.bz2
08192007.tar.bz2 08222007.tar.bz2 08112007.tar.bz2
08142007.tar.bz2 08172007.tar.bz2 08202007.tar.bz2
08232007.tar.bz2 08122007.tar.bz2 08152007.tar.bz2
08182007.tar.bz2 08212007.tar.bz2

Or, perhaps I want to pick out only a couple of the files, but all in one command.

[mshade@opteron mysql]$ ls -l 08{10,15,23}*
-rw-r--r-- 1 root root 34308592 Aug 10 04:03 08102007.tar.bz2
-rw-r--r-- 1 root root 35263571 Aug 15 04:03 08152007.tar.bz2
-rw-r--r-- 1 root root 36662944 Aug 23 04:03 08232007.tar.bz2


Learning to use the command line efficiently can really speed up your work. I used to find myself creating awkward for loops in many situations like this. It was intuitive at the time, but bash’s built in capabilities make it much easier and simpler.

Posted in blog entries