>>>>> On Wed, 12 Apr 2017 08:03:32 +0100 (BST), Tom Yates said:
> 
> I've got a fairly big filesystem (3TB, 15M files) of which I want to 
> (test) restore a part.  I know that if the backend DB is slow the 
> "Building file list" stage can take some time, but I have it striped over 
> a 5-SAS-disc RAID-0, and this step takes only about eight minutes.
> 
> The problems start once I navigate to the directory I want restored 
> (which admittedly contains the bulk of the files and about half the total 
> space), and do an "add home".
> 
> The current job has been stuck on this step for over 15 hours, now.  When 
> I strace bacula-dir I see a lot of:
> 
> [pid 26711] write(6, "P\0\0\0\3SELECT FilenameId FROM File"..., 84) = 84
> [pid 26711] read(6, "\1\0\0\1\1@\0\0\2\3def\6bacula\10Filename\10Fi"..., 
> 16384) = 102
> [pid 26711] poll([{fd=6, events=POLLIN|POLLPRI}], 1, 0) = 0 (Timeout)
> [pid 26711] write(6, "m\0\0\0\3SELECT FileId, LStat, MD5 F"..., 113) = 113
> [pid 26711] read(6, "\1\0\0\1\0030\0\0\2\3def\6bacula\4File\4File\6F"..., 
> 16384) = 249
> [pid 26711] poll([{fd=6, events=POLLIN|POLLPRI}], 1, 0) = 0 (Timeout)
> [pid 26711] write(6, "P\0\0\0\3SELECT FilenameId FROM File"..., 84) = 84
> [pid 26711] read(6, "\1\0\0\1\1@\0\0\2\3def\6bacula\10Filename\10Fi"..., 
> 16384) = 102
> [pid 26711] poll([{fd=6, events=POLLIN|POLLPRI}], 1, 0) = 0 (Timeout)
> [pid 26711] write(6, "m\0\0\0\3SELECT FileId, LStat, MD5 F"..., 113) = 113
> [pid 26711] read(6, "\1\0\0\1\0030\0\0\2\3def\6bacula\4File\4File\6F"..., 
> 16384) = 250
> [pid 26711] poll([{fd=6, events=POLLIN|POLLPRI}], 1, 0) = 0 (Timeout)
> 
> So I presume it's stepping through the built directory tree querying the 
> database about each of these files.  Problem is that any restore that 
> takes ~24 hours just to kick off is not making my clients happy.

Does that file tree have a lot of hard links (I think the add command only
makes those queries for hard links)?  If so, then using Bacula 7 might help
(see "restore optimizespeed" in
http://www.bacula.org/downloads/Bacula-7.4.0/ReleaseNotes).

__Martin

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users