BackupPC-users

Re: [BackupPC-users] Full backup not backing up all files

2014-10-14 10:39:06
Subject: Re: [BackupPC-users] Full backup not backing up all files
From: Mikko Kortelainen <mikko.kortelainen AT techelp DOT fi>
To: Holger Parplies <wbppc AT parplies DOT de>
Date: Tue, 14 Oct 2014 17:30:36 +0300
Holger, thanks for your response.

The smb.conf should be pretty much stock on both machines. Besides
comments and whitespace, the only difference is this:

   client ntlmv2 auth = no

That's set on the Samba 3.5 host but not on the 4.1. Actually it seems
that the default for that has changed from Samba 3 (no) to Samba 4
(yes). I set it to no and ran a full backup again but the result was
no different.

A colleague of mine just told me he did a Samba downgrade from 4.1 to
3.5 and tested a quick full backup which was fine. I haven't tried it
myself yet.

What I did do is I ran a full backup command from the command line
with smbclient 4.1 and it stops here:

           0 (    0,0 kb/s) \MSDOS.SYS
       94720 (  818,6 kb/s) \msizap.exe
       47772 ( 2744,3 kb/s) \NTDETECT.COM
      297072 ( 4917,1 kb/s) \ntldr
NT_STATUS_SHARING_VIOLATION opening remote file \pagefile.sys (\)
NT_STATUS_SHARING_VIOLATION listing \\*
tar: dumped 26297 files and directories
Total bytes written: 17752618496

So it stops after the first NT_STATUS_SHARING_VIOLATION on a file in
the root directory of C$. There's a bug related to it and tagged to
Samba 3.6:

"smbclient TAR omits remainder of directory on access denial."
https://bugzilla.samba.org/show_bug.cgi?id=10605

Perhaps Samba 4.1 suffers from the same bug.

This is pretty bad as the backup is still listed as a good backup,
even though it is far from complete. And the error log can have more
NT_STATUS_SHARING_VIOLATION messages so it is not immediately obvious
that the backup stopped before copying everything it can. It only
omits the rest of the files and subdirectories in the same directory
with that error and continues with the next same or upper level
directory. Being the root of C$ there are no same level or upper level
directories left to back up, so it stops completely.

As time goes by, the incrementals get deleted from disk, and any files
not included in the full backups will be gone, if I have understood
correctly (?). So there will be at most IncrAgeMax days of complete
backups available. The full backups are just partial.

I guess the only thing I can do now is downgrade to Samba 3.5.

It is quite alarming to realize this thing may bite every user out
there with the default packages and settings on any relatively new
distro. And there is no good warning about it. Looking at the error
log, the problem is not immediately obvious. It is obvious only after
you browse your full backups and discover some stuff is not there.

-Mikko


2014-10-14 15:26 GMT+03:00 Holger Parplies <wbppc AT parplies DOT de>:
> Hi,
>
> Mikko Kortelainen wrote on 2014-10-14 12:18:00 +0300 [[BackupPC-users] Full 
> backup not backing up all files]:
>> I have a problem with two BackupPC hosts, yet there's a further third
>> host that is working ok.
>>
>> The problem is that a full backup does not seem to back up all files
>> on Windows hosts. An incremental backs up many more files than a full
>> one, or so it seems, looking at the version history.
>> [...]
>> What could be the problem here? Something to do with the Samba version
>> (3.5 vs. 4.1)?
>
> so it would seem. Or maybe only a changed default configuration. I don't know
> enough about smbclient to guess what it might be, but I know that smb.conf is
> relevant for the smbclient command as well.
>
>> What can I do to diagnose this?
>
> A 'diff' of the two smb.conf files might shed some light, presuming it is
> configuration related. Otherwise, I would expect the old smbclient package
> (along with the same version samba-common) to work on the newer system. You
> have several options for trying the downgrade (***Note: I am assuming you have
> nothing installed on the system that depends on smbclient/samba-common, i.e.
> no samba server, samba configuration tool etc. If in doubt, check with
> 'apt-cache rdepends samba-common smbclient'***):
>
> - download the packages with a browser and install with 'dpkg -i',
> - add a sources.list line for the old distribution on the host(s) with the
>   new one and do something like
>
>     apt-get update
>     apt-get install --reinstall smbclient=xyz samba-common=xyz
>
>   (where 'xyz' is the version of smbclient/samba-common on the old system;
>   'apt-cache policy smbclient' might help figuring things out), or
> - download the packages on the old system with
>
>     apt-get -d install --reinstall smbclient samba-common
>
>   (the '-d' switch means 'download only'). Copy them from
>   /var/cache/apt/archives over to the new host and install with 'dpkg -i'.
>
> You'll probably need to confirm the downgrade for 'apt-get' and possibly
> use a '--force-downgrade' switch on dpkg, but I don't think so.
>
> Presuming downgrading helps and you want to keep things that way, you
> should put the packages on hold (so they won't be inadvertantly
> upgraded at some point), and you should probably browse the changelogs to
> see if there were any important security fixes that the old version misses.
>
> Otherwise, 'apt-get install smbclient samba-common' or simply
> 'apt-get upgrade' should revert things to the new version.
>
> Hope that helps.
>
> Regards,
> Holger
>
> P.S.: There might be some other changed dependencies that I missed. In
>       general, apt-get should take care of things. If you use 'dpkg -i',
>       you can follow that with 'apt-get -f install' to fix dependencies,
>       though in awkward situations this occasionally tends to want to
>       fix things by removing the package you just installed. Always
>       check what 'apt-get' wants to do, and be very sceptical if it wants
>       to remove packages.
>       Bottom line: it's easy enough to attempt the downgrade and simply
>       *not* do it if it turns out to require more than replacing the two
>       packages with older versions.

------------------------------------------------------------------------------
Comprehensive Server Monitoring with Site24x7.
Monitor 10 servers for $9/Month.
Get alerted through email, SMS, voice calls or mobile push notifications.
Take corrective actions from your mobile device.
http://p.sf.net/sfu/Zoho
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

<Prev in Thread] Current Thread [Next in Thread>