ADSM-L

Re: 15,000,000 + files on one directory backup

2005-06-16 14:10:01
Subject: Re: 15,000,000 + files on one directory backup
From: "Stapleton, Mark" <mark.stapleton AT BERBEE DOT COM>
To: ADSM-L AT VM.MARIST DOT EDU
Date: Thu, 16 Jun 2005 13:09:44 -0500
You create 10 separate nodes, one for each mountpoint (drive). You can
then run 10 separate backup threads; each thread has (in theory) 1/10th
as many files to filter through before a backup begins.

It's done that way a lot in such cases.

--
Mark Stapleton (stapleton AT berbee DOT com)
IBM Certified Advanced Deployment Professional
  Tivoli Storage Management Solutions 2005
IBM Certified Advanced Technical Expert (CATE) AIX
Office 262.521.5627

 

>-----Original Message-----
>From: ADSM: Dist Stor Manager [mailto:ADSM-L AT VM.MARIST DOT EDU] On 
>Behalf Of Dearman, Richard
>Sent: Thursday, June 16, 2005 1:06 PM
>To: ADSM-L AT VM.MARIST DOT EDU
>Subject: Re: 15,000,000 + files on one directory backup
>
>Yes, but the long backup time is because of the time it takes the TSM
>client to query the TSM database for backup file candidates and not due
>to the actual movement of files from the client to the TSM server.  So
>how pulling from 10 separate drives increase the query speed.
>
>For instance it takes hours and GB of memory on my client for 
>the client
>to query the TSM server for file info for 15 million files even though
>the actual backup will end up being 11,000 files consisting of 300mb.
>And the backup take over 17+ hours to complete.  This is running from a
>GigE connection with my client with 4 cpus and 6GB of memory 
>although it
>is Win2k.
>
>-----Original Message-----
>From: ADSM: Dist Stor Manager [mailto:ADSM-L AT VM.MARIST DOT EDU] On 
>Behalf Of
>Stapleton, Mark
>Sent: Thursday, June 16, 2005 12:53 PM
>To: ADSM-L AT VM.MARIST DOT EDU
>Subject: Re: 15,000,000 + files on one directory backup
>
>No, because you're pulling data off of 10 separate physical 
>drives. It's
>likely to be significantly faster.
>
>--
>Mark Stapleton (stapleton AT berbee DOT com)
>IBM Certified Advanced Deployment Professional
>  Tivoli Storage Management Solutions 2005
>IBM Certified Advanced Technical Expert (CATE) AIX
>Office 262.521.5627
>
> 
>
>>-----Original Message-----
>>From: ADSM: Dist Stor Manager [mailto:ADSM-L AT VM.MARIST DOT EDU] On 
>>Behalf Of Dearman, Richard
>>Sent: Thursday, June 16, 2005 12:46 PM
>>To: ADSM-L AT VM.MARIST DOT EDU
>>Subject: Re: 15,000,000 + files on one directory backup
>>
>>Wouldn't the backup take close to the same amount of time has 
>using one
>>mount point.  Because the TSM client on that one server still 
>had to go
>>through 15 million files whether its doing it in one session or 10
>>sessions.
>>
>>I am experiencing the same problem of an imaging system.  I am 
>>trying to
>>go the snapshot image route of the 12 millions 300Gb of files and
>>sending them to disk storage pool then off to 3592 tapes nightly.  It
>>will grow to 1Tb of the next year.  I'm not sure how imaging will work
>>on such a large file system.
>>
>>-----Original Message-----
>>From: ADSM: Dist Stor Manager [mailto:ADSM-L AT VM.MARIST DOT EDU] On 
>>Behalf Of
>>TSM_User
>>Sent: Thursday, June 16, 2005 12:32 PM
>>To: ADSM-L AT VM.MARIST DOT EDU
>>Subject: 15,000,000 + files on one directory backup
>>
>>I replied to the list about using windows mount points. A 
>>reply was sent
>>back about it being the opposit of what they wanted. Well I had this
>>thought.
>>
>>If you have an imaging system (or any other application) that can only
>>use one drive letter why not use mount points for that drive.  Have a
>>server with 10 drives with 100 GB each (D: - M:)  Then use 
>mount points
>>to get all 1 TB of space behind the D:\.  The application 
>will then use
>>the D:\ alone.  Mean while you can run the backup on all 10 drives.
>>
>>For details on how to set this up you need to consult the 
>>Microsoft doc.
>>There is plenty on MS's website.
>>
>>Of course if the system is already setup then you'd need to add the
>>drives and mount points.  Then move the folders under those mount
>>points.
>>
>>__________________________________________________
>>Do You Yahoo!?
>>Tired of spam?  Yahoo! Mail has the best spam protection around
>>http://mail.yahoo.com
>>
>>**************************EMAIL DISCLAIMER***************************
>>
>>This email and any files transmitted with it may be 
>>confidential and are
>>intended solely for the use of the individual or entity to 
>>whom they are
>>addressed.  If you are not the intended recipient or the individual
>>responsible for delivering the e-mail to the intended recipient, any 
>>disclosure, copying, distribution or any action taken or 
>omitted to be 
>>taken in reliance on it, is strictly prohibited.  If you have 
>>received this
>>e-mail in error, please delete it and notify the sender or 
>>contact Health 
>>Information Management 312.413.4947.
>>
>> 
>>
>
>**************************EMAIL DISCLAIMER***************************
>
>This email and any files transmitted with it may be 
>confidential and are
>intended solely for the use of the individual or entity to 
>whom they are
>addressed.  If you are not the intended recipient or the individual
>responsible for delivering the e-mail to the intended recipient, any 
>disclosure, copying, distribution or any action taken or omitted to be 
>taken in reliance on it, is strictly prohibited.  If you have 
>received this
>e-mail in error, please delete it and notify the sender or 
>contact Health 
>Information Management 312.413.4947.
>
> 
>