I've got a few clients with rather large filesystems: ~6K dirs, ~2.2M
files, 10 filesystems to the client.
Walking these FS trees, especially given the size of the directories,
can be very slow. I'm noodling with methods that would let me take
some shortcuts, for daily purposes, and then do a normal incremental
less frequently.
Linux has this neat feature 'inotify'. It lets you erect a low-level
FS monitor on a directory or file, and get warnings when something
happens to it.
I started to write my own direct inotify interface, but then found
inotify-tools, which is capable of dumping stuff out with substantial
flexibility.
My current state of experimentation is using an inotifywait
invocation, and the following PERL code thus:
# inotifywait -c -m -r [path] | ./timesplit
which yields a series of timestamped "changes" files. I'll be doing a
periodical
dsmc incr -filelist=/some/path/yadda/timestamped-change-file
in the next phase.
I'm wondering if anyone else is doing something similar, or is
interested in comparing notes.
- Allen S. Rout
-----
#!/usr/bin/perl -- -*-Perl-*-
use warnings;
use strict;
use Getopt::Long;
use Time::CTime;
use FileHandle;
my $outdir = "./out";
my $prefix="changes";
my $formatstring= "%Y-%m-%d--%H-%M";
my $outfile = &newfile() ;
my $last = time;
my $delay = 60;
# Seconds. If this delay doesn't dependably
# result in a change in the formatted time,
# you'll overwrite stuff.
while (<>)
{
my $now = time;
if ( ($now - $last) > $delay )
{
$outfile->close();
$outfile = &newfile();
$last = $now;
}
my $l = &procline($_);
defined ($l) && print $outfile $l;
print STDERR ".";
}
sub newfile
{
$last = time;
my $fname = "$outdir/$prefix-".strftime($formatstring,localtime($last));
print "\n".$fname."\n";
my $fh = new FileHandle(">$fname");
return $fh;
}
sub procline
{
my @fields;
undef @fields;
push(@fields, defined($1) ? $1:$3)
while m/"([^"\\]*(\\.[^"\\]*)*)"|([^,]+)/g; # \\]
my ($path,$op,$fname) = @fields;
# return undef if ($fname =~ /cyrus.(header|index|cache)/);
return "$path$fname";
}
-----
|