Re: [BackupPC-users] pools are showing data but clients have no backups

Hi,

Added some debugging code to Storage/Text.pm sub TextFileWrite, it
used to look like this :

rename("$file.new", "$file") if ( -f "$file.new" );

I changed it to :

        if ( -f "$file.new" ) {
                my $renRet = rename("$file.new", "$file");
                if ($renRet) {
                        print "Rename OK\n";
                }
                else {
                        print "Rename failed $file.new to $file\n";
                }
        }
        else {
                print "failed -f $file.new\n";
        }

This is what BackupPC_fixupBackupSummary returns when I run it as root
for one of the backup hosts where backups has only up to #45 and
backups.new has #46:

    Reading /glfsdist/backuppc1/pc/home_caox/46/backupInfo
    Adding info for backup 46 from backupInfo file
Rename failed /glfsdist/backuppc1/pc/home_caox/backups.new to
/glfsdist/backuppc1/pc/home_caox/backups

[root@gluster1 ~]# ls -l  /glfsdist/backuppc1/pc/home_caox/backups*
-rw-r----- 1 root root 3440 Dec 15 12:41
/glfsdist/backuppc1/pc/home_caox/backups
-rw-r----- 1 root root 3514 Dec 16 14:00
/glfsdist/backuppc1/pc/home_caox/backups.new

Let's try a manual move:

[root@gluster1 ~]# mv /glfsdist/backuppc1/pc/home_caox/backups.new
/glfsdist/backuppc1/pc/home_caox/backups
mv: overwrite `/glfsdist/backuppc1/pc/home_caox/backups'? y
mv: cannot move `/glfsdist/backuppc1/pc/home_caox/backups.new' to
`/glfsdist/backuppc1/pc/home_caox/backups': File exists

hrmm... here's the strace -ff :

stat("/glfsdist/backuppc1/pc/home_caox/backups", y

^ that's me hitting 'y' when it asks if I want to overwrite the file.

{st_mode=S_IFREG|0640, st_size=3440, ...}) = 0
lstat("/glfsdist/backuppc1/pc/home_caox/backups.new",
{st_mode=S_IFREG|0640, st_size=3514, ...}) = 0
lstat("/glfsdist/backuppc1/pc/home_caox/backups",
{st_mode=S_IFREG|0640, st_size=3440, ...}) = 0
stat("/glfsdist/backuppc1/pc/home_caox/backups",
{st_mode=S_IFREG|0640, st_size=3440, ...}) = 0
geteuid()                               = 0
getegid()                               = 0
getuid()                                = 0
getgid()                                = 0
access("/glfsdist/backuppc1/pc/home_caox/backups", W_OK) = 0
rename("/glfsdist/backuppc1/pc/home_caox/backups.new",
"/glfsdist/backuppc1/pc/home_caox/backups") = -1 EEXIST (File exists)
open("/usr/share/locale/locale.alias", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=2528, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = 0x2b6fc324b000
read(3, "# Locale name alias data base.\n#"..., 4096) = 2528
read(3, "", 4096)                       = 0
close(3)                                = 0
munmap(0x2b6fc324b000, 4096)            = 0
open("/usr/share/locale/en_US.UTF-8/LC_MESSAGES/coreutils.mo",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en_US.utf8/LC_MESSAGES/coreutils.mo",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en_US/LC_MESSAGES/coreutils.mo", O_RDONLY) =
-1 ENOENT (No such file or directory)
open("/usr/share/locale/en.UTF-8/LC_MESSAGES/coreutils.mo", O_RDONLY)
= -1 ENOENT (No such file or directory)
open("/usr/share/locale/en.utf8/LC_MESSAGES/coreutils.mo", O_RDONLY) =
-1 ENOENT (No such file or directory)
open("/usr/share/locale/en/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1
ENOENT (No such file or directory)
write(2, "mv: ", 4mv: )                     = 4
write(2, "cannot move `/glfsdist/backuppc1"..., 104cannot move
`/glfsdist/backuppc1/pc/home_caox/backups.new' to
`/glfsdist/backuppc1/pc/home_caox/backups') = 104
open("/usr/share/locale/en_US.UTF-8/LC_MESSAGES/libc.mo", O_RDONLY) =
-1 ENOENT (No such file or directory)
open("/usr/share/locale/en_US.utf8/LC_MESSAGES/libc.mo", O_RDONLY) =
-1 ENOENT (No such file or directory)
open("/usr/share/locale/en_US/LC_MESSAGES/libc.mo", O_RDONLY) = -1
ENOENT (No such file or directory)
open("/usr/share/locale/en.UTF-8/LC_MESSAGES/libc.mo", O_RDONLY) = -1
ENOENT (No such file or directory)
open("/usr/share/locale/en.utf8/LC_MESSAGES/libc.mo", O_RDONLY) = -1
ENOENT (No such file or directory)
open("/usr/share/locale/en/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT
(No such file or directory)
write(2, ": File exists", 13: File exists)           = 13
write(2, "\n", 1
)                       = 1
close(1)                                = 0
exit_group(1)                           = ?

This has got to be some glusterfs weirdness but why? any ideas?

Thanks,
Sabuj

> The file backups.new is written out by a function called BackupInfoWrite
> (which further uses a function called TextFileWrite), which is called
> both in BackupPC_dump, when a backup completes and in BackupPC_link,
> when the linking of the backup completes. The file backups.new is
> written, verified and then renamed to backups. If the verification
> fails, the rename does not occur. I'd suggest checking for permissions
> issues or file system corruption.

> Chris

------------------------------------------------------------------------------
This SF.Net email is sponsored by the Verizon Developer Community
Take advantage of Verizon's best-in-class app development support
A streamlined, 14 day to market process makes app distribution fast and easy
Join now and get one step closer to millions of Verizon customers
http://p.sf.net/sfu/verizon-dev2dev 
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/