ADSM-L

Re: [ADSM-L] Linux inotify to identify changed stuff?

2010-04-22 12:37:56
Subject: Re: [ADSM-L] Linux inotify to identify changed stuff?
From: km <km AT GROGG DOT ORG>
To: ADSM-L AT VM.MARIST DOT EDU
Date: Thu, 22 Apr 2010 18:18:14 +0200
Really nice!

Inotify has been a part of the kernel since 2.6.13 and is supported
on all RHEL 5 releases and atleast SLES 10+ so this should work on any
modern Linux.

-km

On 21/04, Allen S. Rout wrote:
> I've got a few clients with rather large filesystems: ~6K dirs, ~2.2M
> files, 10 filesystems to the client.
>
> Walking these FS trees, especially given the size of the directories,
> can be very slow.  I'm noodling with methods that would let me take
> some shortcuts, for daily purposes, and then do a normal incremental
> less frequently.
>
> Linux has this neat feature 'inotify'. It lets you erect a low-level
> FS monitor on a directory or file, and get warnings when something
> happens to it.
>
> I started to write my own direct inotify interface, but then found
> inotify-tools, which is capable of dumping stuff out with substantial
> flexibility.
>
>
> My current state of experimentation is using an inotifywait
> invocation, and the following PERL code thus:
>
>
> #  inotifywait  -c -m   -r  [path] | ./timesplit
>
> which yields a series of timestamped "changes" files.  I'll be doing a
> periodical
>
> dsmc incr -filelist=/some/path/yadda/timestamped-change-file
>
> in the next phase.
>
>
>
> I'm wondering if anyone else is doing something similar, or is
> interested in comparing notes.
>
>
> - Allen S. Rout
>
>
>
> -----
> #!/usr/bin/perl -- -*-Perl-*-
>
> use warnings;
> use strict;
> use Getopt::Long;
> use Time::CTime;
> use FileHandle;
>
>
>
> my $outdir = "./out";
>
> my $prefix="changes";
>
> my $formatstring= "%Y-%m-%d--%H-%M";
>
> my $outfile = &newfile() ;
>
> my $last = time;
>
> my $delay = 60;
> # Seconds.  If this delay doesn't dependably
> # result in a change in the formatted time,
> # you'll overwrite stuff.
>
>
> while (<>)
> {
>     my $now = time;
>
>     if ( ($now - $last) > $delay  )
>     {
>         $outfile->close();
>         $outfile = &newfile();
>         $last = $now;
>     }
>
>     my $l = &procline($_);
>     defined ($l) && print $outfile  $l;
>     print STDERR ".";
> }
>
>
> sub newfile
> {
>     $last = time;
>     my $fname = "$outdir/$prefix-".strftime($formatstring,localtime($last));
>     print "\n".$fname."\n";
>     my $fh = new FileHandle(">$fname");
>     return $fh;
> }
>
>
>
>
>
> sub procline
> {
>     my @fields;
>     undef @fields;
>     push(@fields, defined($1) ? $1:$3)
>         while m/"([^"\\]*(\\.[^"\\]*)*)"|([^,]+)/g; # \\]
>
>                               my ($path,$op,$fname) = @fields;
>
> #      return undef if ($fname =~ /cyrus.(header|index|cache)/);
>
>                               return "$path$fname";
>                           }
>
> -----

<Prev in Thread] Current Thread [Next in Thread>