Re: unix sparse file backup?

I seem to remember someone posting a link on the list that included
information on the hows and whys of reading/re-creating  sparse files
and algorithms to do so. I just had a quick search but couldn't spot
it...


Matt.

_-'-_
  -|-


-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:ADSM-L AT VM.MARIST DOT EDU] On Behalf Of
Jurjen Oskam
Sent: Wednesday, January 26, 2005 8:30 AM
To: ADSM-L AT VM.MARIST DOT EDU
Subject: Re: unix sparse file backup?

On Tue, Jan 25, 2005 at 03:56:00PM -0700, Andrew Raibeck wrote:

> > > file, here is allocation map". My understanding is that during
restore,
> > > TSM analyzes the data for lengthy patterns of zeroes in the data
stream,
> > > and that is how MAKESPARSEFILE works; but TSM does not do that
analysis
> > > during the backup.
        [...]
> The TSM server is not actively "aware" of a file's attributes. The
> "sparseness" of the file is determined at restore time, after the data
has
> been uncompressed, but before it is written to the disk.

I've just been testing this a bit. I've created two files, both 10M
large. Both are entirely filled with zeros. One is sparse, one isn't:

$ dd if=/dev/zero of=test-real bs=1024 count=10240
10240+0 records in.
10240+0 records out.
$ dd if=/dev/zero of=test-sparse bs=1024 count=1 seek=10239
1+0 records in.
1+0 records out.
$ ls -l test-real test-sparse
-rw-r--r--   1 joskam   staff      10485760 Jan 26 08:52 test-real
-rw-r--r--   1 joskam   staff      10485760 Jan 26 08:52 test-sparse
$ cmp test-real test-sparse
$ du -k test-real test-sparse
10240   test-real
4       test-sparse

Then I did (dsmc output compacted):

$ dsmc sel test-real test-sparse
Normal File-->        10,485,760 /home/joskam/test-real [Sent]
Normal File-->        10,485,760 /home/joskam/test-sparse [Sent]
Total number of bytes transferred:    20.00 MB

$ rm test-real test-sparse
$ ls -l test-real test-sparse
ls: 0653-341 The file test-real does not exist.
ls: 0653-341 The file test-sparse does not exist.
$ dsmc rest -makesparsefile=yes test-real
Restoring      10,485,760 /home/joskam/test-real [Done]
Total number of bytes transferred:    10.00 MB

$ dsmc rest -makesparsefile=yes test-sparse
Restoring      10,485,760 /home/joskam/test-sparse [Done]
Total number of bytes transferred:    10.00 MB


If what you're stating above is true, TSM would have no knowledge of
whether a file was originally sparse or not. But it does:

$ du -k test-real test-sparse
10240   test-real
4       test-sparse

Both the above files were restored with MAKESPARSEFILE=YES, but only
the file that originally was sparse is restored as such.

I repeated this with 512 MB files, and got the exact same results.
With these larger files however, it was easily observable that
the sparse file and the normal file took the same amount of time to
be sent to the TSM server. This also applied to restoring.

So it would seem that sparse files are indeed sent to and stored on
the TSM server without any "holes". However, it would also seem that
the TSM server does indeed know whether a file was sparse or not.

I did not test whether sparse files are restored with the same layout,
i.e. whether the "holes" in restored sparse files are in the same
place as the "holes" in the original files.

--
Jurjen Oskam
  "E-mail has just erupted like a weed, and instead of considering
  what to say when they write, people now just let thoughts drool
  out onto the screen." - R. Craig Hogan


___________________________ Disclaimer Notice __________________________
This message and any attachments are confidential and should only be read by 
those to whom they are addressed. If you are not the intended recipient, please 
contact us, delete the message from your computer and destroy any copies. Any 
distribution or copying without our prior permission is prohibited.

Internet communications are not always secure and therefore Powergen Retail 
Limited does not accept legal responsibility for this message. The recipient is 
responsible for verifying its authenticity before acting on the contents. Any 
views or opinions presented are solely those of the author and do not 
necessarily represent those of Powergen Retail Limited. 

Registered addresses:

Powergen Retail Limited, Westwood Way, Westwood Business Park, Coventry, CV4 
8LG.
Registered in England and Wales No: 3407430

Telephone +44 (0) 2476 42 4000
Fax +44 (0) 2476 42 5432