ADSM-L

Re: Sub-File Backup - 1 GB Limit

2003-01-08 20:06:49
Subject: Re: Sub-File Backup - 1 GB Limit
From: Jim Smith <smithjp AT US.IBM DOT COM>
To: ADSM-L AT VM.MARIST DOT EDU
Date: Wed, 8 Jan 2003 17:04:48 -0800
Subfilecachesize could be raised; please submit a requirement and let us
know what types of cache sizes you need.  The cache size is not bounded by
the 32-bit differencing image limit.

System objects and subfile processing would be very difficult.  I would
suggest pushing a requirement for incremental processing of system files,
which are the files that get resent on every system object backup that are
probably causing the majority of your woes.   That would probably
alleviate your problem and be a more applicable solution to other problems
in this area.

Thanks,
Jim

Thanks,
Jim

J.P. (Jim) Smith
TSM Client Development
smithjp AT us.ibm DOT com


Hi Jim

postings from development are always welcome.

Beeing curious I (mis)use your readiness and ask
you prior to opening a requirement:

Are you limited with 32-bit addressing
with SubFileCachesize with its maximum of 1 GB as well,
or would it be simple for you to raise this limit significantly higher?

The business case is ability to backup (almost) whole filesystems
on small file servers over leased lines with limited throughput.

And - any would it be troublesome to apply differencing technology for
system objects backup?
The business case is ability to backup enormous system objects from
servers
with microsoft systems over leased lines with limited throughput.

best regards
Juraj Salak, Asamer Holding




> -----Original Message-----
> From: Jim Smith [mailto:smithjp AT US.IBM DOT COM]
> Sent: Monday, January 06, 2003 10:00 PM
> To: ADSM-L AT VM.MARIST DOT EDU
> Subject: Re: Sub-File Backup - 1 GB Limit
>
>
> Tim,
>
> Actually, two different behaviors based on two-different
> problems.   Files
> that start less then 2 GB but grow > 2 GB will continue to use subfile
> backup as long as the other requirements for the base file
> (i.e., the file
> on the client cache) is still valid.  The limiting factor here is that
> there is only 32-bit support in the differencing subsystem that we are
> using.  We chose 2 GB on the onset (instead of 4 GB) as the
> limit to avoid
> any possible boundary problems near the 32-bit addressing
> limit and also
> because this technology was aimed at the mobile market (read:
> who is going
> to have files on their laptops > 2 GB).  I understand that there are
> several shops that use this technology beyond the laptop environment.
> Ultimately, the solution is to have a 64-bit subsystem in
> place so that we
> can go beyond 4 GB.  I suggest a requirement to Tivoli if this is
> important to your shop.
>
> The low-end limit (1024 bytes) was due to some strange behavior with
> really small files, e.g., if a file started out at 5 k and then was
> truncated to 8 bytes.  The solution was to just send the
> entire file if
> the file fell below the 1k threshold.  We can get away with resending
> these small files because ... they are small files!  It is
> probably a wash
> to resend or to try to correctly send a delta file in this case.
>
> Hope this helps.
>
> - Jim
>
> J.P. (Jim) Smith
> TSM Client Development
> smithjp AT us.ibm DOT com
>
>
> Thanks Jim!
>
> I was confusing "size of base file falls < 1024" with 1 GB!
>
> So if a file starts at less than 2 GB but then grows bigger
> than 2GB it
> will
> no longer be eligible?  Similar if a file falls below 1024 bytes?
>
> Thanks again!
>
> Tim
>
> -----Original Message-----
> From: Jim Smith [mailto:smithjp AT US.IBM DOT COM]
> Sent: January 3, 2003 4:36 PM
> To: ADSM-L AT VM.MARIST DOT EDU
> Subject: Re: Sub-File Backup - 1 GB Limit
>
> Tim,
>
> Actually, the subfile limit is 2 Gig; if a file size > 2 Gig
> then TSM will
> not bother to copy the base file to the client cache, so it
> could not be a
> candidate for subfile processing on a subsequent backup.
>
> Thanks,
> Jim
>
> J.P. (Jim) Smith
> TSM Client Development
> smithjp AT us.ibm DOT com
>
>
> Last August, Jim Smith wrote:
>
>
> >Today the client code will send a new base file in these cases:
> >
> >- if the last time the detla was taken, the ratio of
> delta:base is > .40
> >- digital signature of base is incorrect or doesn't match
> signature on
> >server
> >- base entry on client cache is dirty, i.e., never committed on the
> server
> >- size of base file falls < 1024
> >- file is excluded from subfile backup processing, i..e,
> exclude.subfile
> >- file is encrypted by Windows efs
>
>
> I just want to double check, does this mean that there is
> still the 1GB
> limit for subfile backup?  Ie. If a file is > 1 GB, is it
> ineligible for
> subfile processing?
>
> Thanks,
>
> Tim Rushforth
> City of Winnipeg
>