Re: unix sparse file backup?

On Tue, Jan 25, 2005 at 03:56:00PM -0700, Andrew Raibeck wrote:

> > > file, here is allocation map". My understanding is that during restore,
> > > TSM analyzes the data for lengthy patterns of zeroes in the data stream,
> > > and that is how MAKESPARSEFILE works; but TSM does not do that analysis
> > > during the backup.
        [...]
> The TSM server is not actively "aware" of a file's attributes. The
> "sparseness" of the file is determined at restore time, after the data has
> been uncompressed, but before it is written to the disk.

I've just been testing this a bit. I've created two files, both 10M
large. Both are entirely filled with zeros. One is sparse, one isn't:

$ dd if=/dev/zero of=test-real bs=1024 count=10240
10240+0 records in.
10240+0 records out.
$ dd if=/dev/zero of=test-sparse bs=1024 count=1 seek=10239
1+0 records in.
1+0 records out.
$ ls -l test-real test-sparse
-rw-r--r--   1 joskam   staff      10485760 Jan 26 08:52 test-real
-rw-r--r--   1 joskam   staff      10485760 Jan 26 08:52 test-sparse
$ cmp test-real test-sparse
$ du -k test-real test-sparse
10240   test-real
4       test-sparse

Then I did (dsmc output compacted):

$ dsmc sel test-real test-sparse
Normal File-->        10,485,760 /home/joskam/test-real [Sent]
Normal File-->        10,485,760 /home/joskam/test-sparse [Sent]
Total number of bytes transferred:    20.00 MB

$ rm test-real test-sparse
$ ls -l test-real test-sparse
ls: 0653-341 The file test-real does not exist.
ls: 0653-341 The file test-sparse does not exist.
$ dsmc rest -makesparsefile=yes test-real
Restoring      10,485,760 /home/joskam/test-real [Done]
Total number of bytes transferred:    10.00 MB

$ dsmc rest -makesparsefile=yes test-sparse
Restoring      10,485,760 /home/joskam/test-sparse [Done]
Total number of bytes transferred:    10.00 MB


If what you're stating above is true, TSM would have no knowledge of
whether a file was originally sparse or not. But it does:

$ du -k test-real test-sparse
10240   test-real
4       test-sparse

Both the above files were restored with MAKESPARSEFILE=YES, but only
the file that originally was sparse is restored as such.

I repeated this with 512 MB files, and got the exact same results.
With these larger files however, it was easily observable that
the sparse file and the normal file took the same amount of time to
be sent to the TSM server. This also applied to restoring.

So it would seem that sparse files are indeed sent to and stored on
the TSM server without any "holes". However, it would also seem that
the TSM server does indeed know whether a file was sparse or not.

I did not test whether sparse files are restored with the same layout,
i.e. whether the "holes" in restored sparse files are in the same
place as the "holes" in the original files.

--
Jurjen Oskam
  "E-mail has just erupted like a weed, and instead of considering
  what to say when they write, people now just let thoughts drool
  out onto the screen." - R. Craig Hogan