BackupPC-users

[BackupPC-users] [OT] Deleting millions of files (was: Re: backups don't complete - don't know why)

2009-08-15 10:37:30
Subject: [BackupPC-users] [OT] Deleting millions of files (was: Re: backups don't complete - don't know why)
From: Holger Parplies <wbppc AT parplies DOT de>
To: Kanwar Ranbir Sandhu <m3freak AT thesandhufamily DOT ca>
Date: Sat, 15 Aug 2009 16:34:45 +0200
Hi,

Kanwar Ranbir Sandhu wrote on 2009-08-14 11:46:02 -0400 [Re: [BackupPC-users] 
backups don't complete - don't know why]:
> [...]
> I found a gnome mplayer cache dir in my home dir which
> had MILLIONS of files in them. [...]
> 
> It took several days of running "find . -type f -exec rm -f '{}' +" off
> and on to delete the files, before backuppc was finally able to
> backup /home.

yes, forking a process for each of millions of files will take virtually
forever. Next time, you can cut that down to a fraction with something like

        find . -type f -print0 | xargs -0 rm

or

        find . -type f -print0 | perl -n0e unlink

or even

        find2perl . -type f -exec rm {} \; | perl

(which special-cases the 'rm' to use Perl unlink and is probably the fastest
version).

If you want to see what's going on (at the cost of slowing it down a bit),

        find . -type f -print0 | perl -lp0e unlink

or
        find2perl . -type f -print -exec rm {} \; | perl

Finally, if you simply want to delete the whole substructure, use good old

        cd ..; rm -r name_of_directory

(add a '-v' to see what's being removed and a '-f' if you need it).

You'll want to be careful with all of the commands if you use '.' instead of a
full (or at least relative) path, because it's really easy to type them into
the wrong shell window, for instance ...

Note that a fraction of "several days" may still be long (unlinking millions
of files simply takes time), but the keyword is "fraction". The longer your
original task took, the more time you will save.

Regards,
Holger

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day 
trial. Simplify your report design, integration and deployment - and focus on 
what you do best, core application coding. Discover what's new with 
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

<Prev in Thread] Current Thread [Next in Thread>