ADSM-L

Re: ADSM with Windows 2003 Server and SFU 3.5

2003-10-10 15:07:20
Subject: Re: ADSM with Windows 2003 Server and SFU 3.5
From: Remco Post <r.post AT SARA DOT NL>
To: ADSM-L AT VM.MARIST DOT EDU
Date: Fri, 10 Oct 2003 21:05:23 +0200
Now, this is about the first response that makes sense.

On Friday, Oct 10, 2003, at 20:26 Europe/Amsterdam, Andrew Raibeck wrote:

True, TSM (like many other Win32 applications) does not support
case-sensitive file names.

It isn't that we are willing to admit that such files exist; of course
they exist. Nor is that we are "unwilling" to back up files supported by
NTFS.

TSM, like any other application, relies on operating system API functions
to operate on OS-specific elements (such as NTFS file systems and the
files that reside thereon). NTFS supports the existence of case-sensitive file names, but there are limitations to the degree of support afforded by
the OS API functions. Some of the issues involved:

- TSM needs to manage case-sensitive file names is differently than
case-insensitive names (hopefully that is obvious). For non-Unix file
names, NTFS is case-preserving, but case-insensitive. Thus 'README.TXT'
and 'readme.txt' are the same file. TSM must operate on this file so that
if the name changes from 'README.TXT' to 'readme.txt' between backup
versions, TSM treats it as the same file. For case-sensitive file names,
TSM must treat 'README.TXT' and 'readme.txt' as different files.


What you are saying is that because Windows allows applications to behave case-insensitive (basically, like any 20 year old dos app) TSM must treat readme.txt as the same file as README.TXT, just because there are loads of applications that still behave that way, and basically if you open a file giving the wrong case but the right name, windows co-operates.

- Most Win32 applications are not case-sensitive with regard to file names
(as noted below, there are some Windows API functions that support
case-sensitive file names). Windows also provides a POSIX subsystem that
allows POSIX applications to create case-sensitive names.

- Case-sensitive file names can coexist on the same volume with
case-insensitive names. This means that at the NTFS volume level, an
application can not make any blanket assumptions about the
case-sensitivity of all files on the volume.


True. TSM probably (I never checked) restores case as it was backed-up.


-  There is nothing in a file's attributes to indicate the case
sensitivity. Therefore an application can not make any assumptions about
the case-sensitivity of a given file.

- While some Windows file I/O functions support case-sensitive file names, others do not. For example, CreateFile(), which is used to open files, has
support for case-sensitive file names; on the other hand,
GetFileAttributes() (used to obtain a file's attributes) and
GetFileSecurity() (used to obtain a file's security attributes) do not
support case-sensitive file names. Thus if you have files 'readme.txt' and 'README.TXT', we would not be able to obtain file attribute and security
information for both files.


which is a major shortcoming of windows, or the particular API TSM uses. I guess you are right in saying that this is out of IBMs scope to fix.

- Since, as stated above, TSM needs to manage case-sensitive file names
differently than case-insensitive file names, TSM *must* be able to
distinguish case-sensitive names from case-insensitive names. Since NTFS
allows the existence of *both*, but provides no functionality to
distinguish them, it is not possible for TSM to distinguish them, either.


This only makes sense if the case of a filename is not preserved during use. I agree that, with TSMs retention policies, we do want to be able to match different versions of one file as close to the useres perception as possible. Since the application changes the case in the name at more or less random occasion, TSM must indeed not distinguish.

In sum, while NTFS may allow case-sensitive file names to exist, the
Windows operating system itself does not provide the functionality
necessary to fully support these files. In fact, you will find that TSM is not the only application to have difficulty with case-sensitive files. For
example, some basic OS functions like DIR or basic applications like
NOTEPAD.EXE do not handle case-sensitive file names, either. Using some
case-sensitive names I generated, the following is output from a sample
session that demonstrates some odd behavior (note that "posix.exe" is a
tool I created to generate case-sensitive file names):


Basically, what you are saying is that, if TSM had the API to properly back-up sace-sensitive, you might not want to do so because most windows applications are case-insensitive. So basically this boils down to a big dilemma, do we want to treat a change in the filename the windows way or the unix way? Unfortunately for the few of us unlucky enough to suffer sfu, that is about the only case where the unix way is better....

Now if TSM was like other back-up solutions, and we didn't distinguish between old versions of changed files or deleted files, ther would not be a problem (apart from the fact that windows sucks...)


==================================================
C:\POSIX>posix README.TXT ReadMe.txt readme.txt

C:\POSIX>dir
 Volume in drive C is Local Disk
 Volume Serial Number is B022-6D5D

 Directory of C:\POSIX

10/10/2003  10:47       <DIR>          .
10/10/2003  10:47       <DIR>          ..
10/10/2003  10:47                    0 README.TXT
10/10/2003  10:47                    0 ReadMe.txt
10/10/2003  10:47                    0 readme.txt
               3 File(s)              0 bytes
               2 Dir(s)   3,427,893,248 bytes free

C:\POSIX>echo Upper case name > README.TXT

C:\POSIX>echo Lower case name > readme.txt

C:\POSIX>echo Mixed case name > ReadMe.txt

C:\POSIX>dir
 Volume in drive C is Local Disk
 Volume Serial Number is B022-6D5D

 Directory of C:\POSIX

10/10/2003  10:47       <DIR>          .
10/10/2003  10:47       <DIR>          ..
10/10/2003  10:47                   18 README.TXT
10/10/2003  10:47                    0 ReadMe.txt
10/10/2003  10:47                    0 readme.txt
               3 File(s)             18 bytes
               2 Dir(s)   3,427,893,248 bytes free

C:\POSIX>type README.TXT
Mixed case name
Mixed case name
Mixed case name

C:\POSIX>type "README.TXT"
Mixed case name
Mixed case name
Mixed case name

C:\POSIX>type "ReadMe.txt"
Mixed case name
Mixed case name
Mixed case name

C:\POSIX>del "readme.txt"

C:\POSIX>dir
 Volume in drive C is Local Disk
 Volume Serial Number is B022-6D5D

 Directory of C:\POSIX

10/10/2003  10:48       <DIR>          .
10/10/2003  10:48       <DIR>          ..
               0 File(s)              0 bytes
               2 Dir(s)   3,427,893,248 bytes free

C:\POSIX>
==================================================

With all of that said, we are investigating ways to make TSM behave better
than it does today, but I do not see a solution that fully supports
case-sensitive names in the near- or medium-term, as we would need
enhancements to the Windows APIs to facilitate such support.

Regards,

Andy

Andy Raibeck
IBM Software Group
Tivoli Storage Manager Client Development
Internal Notes e-mail: Andrew Raibeck/Tucson/IBM@IBMUS
Internet e-mail: storman AT us.ibm DOT com

The only dumb question is the one that goes unasked.
The command line is your friend.
"Good enough" is the enemy of excellence.




Remco Post <r.post AT SARA DOT NL>
Sent by: "ADSM: Dist Stor Manager" <ADSM-L AT VM.MARIST DOT EDU>
10/10/2003 06:29
Please respond to "ADSM: Dist Stor Manager"

        To:     ADSM-L AT VM.MARIST DOT EDU
        cc:
        Subject:        Re: ADSM with Windows 2003 Server and SFU 3.5


On Fri, 10 Oct 2003 15:06:40 +0200
Gregor Küpper <kuepper AT INFORMATIK.UNI-BONN DOT DE> wrote:

Hi,

we have a installation with a Windows 2003 Cluster and Microsoft
Services for UNIX SFU 3.5
In SFU we enable case sensitive filename processing. ( AAA.txt is not
the same as aaa.txt )
We are able to backup these files, but cannot restore them.
The error message is: file already exist
Who can help us?

Regards,

Gregor K|pper

IBM can. As soon as Tivoli is ready to admit that it is a feature of the NTFS filesystem to be not only case preserving, but also case-sensitive,
and
they are ready to admit that is is their job to be able to backup and
restore all files that can legally excist on a NTFS filesystem.

Basically IBM claims WAD, NTFS is not supposed to be used in a case
sensitive way, so we don't support that....

--
Met vriendelijke groeten,

Remco Post

SARA - Reken- en Netwerkdiensten http://www.sara.nl High Performance Computing Tel. +31 20 592 8008 Fax. +31 20 668 3167

"I really didn't foresee the Internet. But then, neither did the computer
industry. Not that that tells us very much of course - the computer
industry
didn't even foresee that the century was going to end." -- Douglas Adams

--
Met vriendelijke groeten,

Remco Post

SARA - Reken- en Netwerkdiensten                      http://www.sara.nl
High Performance Computing  Tel. +31 20 592 8008    Fax. +31 20 668 3167

"I really didn't foresee the Internet. But then, neither did the computer industry. Not that that tells us very much of course - the computer industry
didn't even foresee that the century was going to end." -- Douglas Adams