Hi,
Adam Goryachev wrote on 2013-10-31 09:04:48 +1100 [Re: [BackupPC-users] Disk
space used far higher than reported pool size]:
> On 31/10/13 07:51, Holger Parplies wrote:
> > [...]
> > Aside from that, I would think it might be worth the effort of determining
> > whether all hosts are affected or not (though I can't really see why there
> > should be a difference between hosts). If some aren't, you could at least
> > keep their history.
> I suspect at least some hosts OR some backups are correct, or else OP
> wouldn't have anything in the pool.
as I understand it, the backups from before the change from smb to rsyncd are
linked into the pool. Since the change, some or all are not. Whether the
change of XferMethod has anything to do with the problem or whether it
coincidentally happened at about the same point in time remains to be seen.
I still suspect the link to $topDir as cause, and BackupPC_link is independent
of the XferMethod used (so a change in XferMethod shouldn't have any influence).
> [...] you might want to look at one individual host like this:
> du -sm /backup/pool /backup/cpool /backup/pc/host1/*
>
> This should be a *lot* quicker than the previous du command, and also
> should show minimal disk usage for each backup for host1. It is quicker
> because you are only looking at the set of files for the pool, plus one
> host.
Just keep in mind that *incrementals* might be small even if not linked to
pool files.
Oh, and there is still another method that is *orders of magnitude* faster:
look into the log file(s), or even at the *size* of the log files. If it
happens every day, for each host, it shouldn't be hard to find. You can even
write a Perl one-liner to show you which hosts it happens for (give me a
sample log line and I will).
If the log files show nothing, we're back to finding the problem, but I doubt
that. You can't "break pooling" by copying, as was suggested. Yes, you get
independent copies of files, and they might stay independent, but changed
files should get pooled again, and your file system usage wouldn't continue
growing in such a way as it seems to be. If pooling is currently "broken",
there's a reason for that, and there should be log messages indicating
problems.
> PS, at this stage, you may want to look at the recent thread regarding
> disk caches, and caching directory entries instead of file contents. It
> might help with all the directory based searches you are doing to find
> the problem. Long term you may (or not) want to keep the settings.
Yes, but remember that for a similarly sized pool it used up about 32 GB of
96 GB available memory. If you can do your investigation on a reasonably idle
system (i.e. not running backups, without long pauses), you should get all the
benefits of caching your amount of memory allows without any tuning. And even
tuning won't let you hold 32 GB of file system metadata in 4 GB of memory :-).
It all depends on file count and hardware memory configuration.
Regards,
Holger
------------------------------------------------------------------------------
Android is increasing in popularity, but the open development platform that
developers love is also attractive to malware creators. Download this white
paper to learn more about secure code signing practices that can help keep
Android apps secure.
http://pubads.g.doubleclick.net/gampad/clk?id=65839951&iu=/4140/ostg.clktrk
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
|
