Best way to setup CIFS (unc) backup of a NAS via TSM?

combato

ADSM.ORG Member
Joined
Mar 22, 2012
Messages
147
Reaction score
1
Points
0
Location
Sweden
PREDATAR Control23

Hi Pro's!

I have some data on a NAS filer (netapp) where I need to save the backups for 25 years. Today I have solved this with NDMP backup and datamovers.
After I have spoken to some guys I have Heard that if I need to restore the data in 25 years from now I need to have a netapp system!! (Is this true??)
This made me worried.

I therefore have tried to backup my filer via CIFS (unc paths). The problem is that I have ~100 TB of data, located on ~50 volumes. How can I setup this in the best way?

One node for each volume share as I schedule with the objects option in a client schedule? (not so funny with fifty nodes)
One node for all volumes (yes please) with many schedules and several volumes specified in the client schedule via objects option?

Any other way of setting this up with the best performance and less administration?

/C
 
PREDATAR Control23

If the intent is to restore data to its original state, then YES you need a Netapp to do that.

This is why I stay away from NDMP and Netapp (if I can).

If I were you, I would dump files on a disk and backup normally.
 
PREDATAR Control23

Yes, the thing now is to setup the CIFS backup in the best way. Have so far tried to backup all shares volumes via one single node but it takes ages.

How can I restore NDMP data taken from a Netapp to Another location and/or system?
 
PREDATAR Control23

Short and easy answer: you need a Netapp device.

Long way: backup the way I posted above and you will never need a Netapp device. Again, I stay away from NDMP and Netapp!
 
PREDATAR Control23

Ok, thanks!
My problem is that all our storage systems worldwide are based on netapp technology so I have no choice.

I'll continue to run my CIFS backup and try to tune my network and TSM server so I can receive data faster. Right now I have an average network speed at 15 MB/s on a 1 Gbps connection. :(
 
PREDATAR Control23

Hello,
to me it makes not such a big difference specifying 50 nodes and 50 schedules than to specifying only one node and 50 schedules. Perhaps scripting may help creating the commands and then "macro" the resulting command file.
I do netapp cifs backups, ~500 TB now with ~20 volumes in 20 nodes as agents to one proxynode target and 20 schedules using snapdiff (netapp snapshots for volumes have to be enabled). Sometimes snapshot processing fails in various volumes. For these a so called "full incremental" backup will be started which creates a new base snapshot so the next backup is a snapdiff again. In a full inc the TSM client API does the process of finding differences. In snapdiff the netapp will do. Backups are incrementals forever unlike NDMP where backups are fulls and diffs. Finding differences in 100 mio files takes about 15 to 20 hours in TSM, but just around 10 minutes with snapdiff.
In schedule's options a put: -snapdiff (for normal snapdiff processing) or -snapdiff -createnewbase (for a full inc, put in a onetime scheduler which will be activated in case of trouble). I also have an -asnodename - Option since I use PROXYNODE...
In the objects-field I put the object to backup up. E.g. \\nodename\unc_name
Probably one should perform "full incs" on a regular basis, since snapshot processing not always knows about all differences the TSM client would find...
Have a nice day,
Friedrich
 
PREDATAR Control23

In schedule's options a put: -snapdiff (for normal snapdiff processing) or -snapdiff -createnewbase (for a full inc, put in a onetime scheduler which will be activated in case of trouble). I also have an -asnodename - Option since I use PROXYNODE...
In the objects-field I put the object to backup up. E.g. \\nodename\unc_name
Probably one should perform "full incs" on a regular basis, since snapshot processing not always knows about all differences the TSM client would find...
Have a nice day,
Friedrich

Hi Fredrich!
Thank you for your answer. I have been off the TSM work for some time (long time), but now started up where I left off and the same problem exists. I today tried to run a client scheduled backup with the following options:
tsm: TSMSERV1>q sch 1_YEARS CIFS_TEST_SCH f=d

Policy Domain Name: 10_YEARS
Schedule Name: CIFS_TEST_SCH
Description:
Action: Incremental
Subaction:
Options: -snapdiff
Objects: "\\nassrv6789\filevolume3$\"
Priority: 5
Start Date/Time: 09/09/2010 10:33:00
Duration: 1 Minute(s)
Maximum Run Time (Minutes): 0
Schedule Style: Enhanced
Period:
Day of Week: Any
Month: Feb
Day of Month: 8
Week of Month: Any
Expiration:
Last Update by (administrator): admin
Last Update Date/Time: 02/08/2017 10:32:46
Managing profile:

tsm: TSMSERV1>

I have a windows 2012 server as I use as node that I have assigned to the above schedule.
The error I get is:

2017-02-08 10:33:59 ------------------------------------------------------------
2017-02-08 10:33:59 Schedule Name: CIFS_TEST_SCH
2017-02-08 10:33:59 Action: Incremental
2017-02-08 10:33:59 Objects: "\\nassrv6789\filevolume3$\"
2017-02-08 10:33:59 Options: -snapdiff
2017-02-08 10:33:59 Server Window Start: 10:33:00 on 2017-02-08
2017-02-08 10:33:59 ------------------------------------------------------------
2017-02-08 10:33:59
Executing scheduled command now.
2017-02-08 10:34:01 --- SCHEDULEREC OBJECT BEGIN CIFS_TEST_SCH 2017-02-08 10:33:00
2017-02-08 10:34:01 ANS1670E The file specification is not valid. Specify a valid Network Appliance or N-Series NFS (AIX, Linux) or CIFS (Windows) volume.
2017-02-08 10:34:02 --- SCHEDULEREC STATUS BEGIN
2017-02-08 10:34:02 ANS5283E The operation was unsuccessful.

What might be wrong???

BR /C
 
PREDATAR Control23

The trailing backslash

Unfortunately did this not help:

2017-02-08 11:57:13 ANS2831E Incremental by snapshot difference cannot be performed on '\\nassrv6789\filevolume3$' as it is of type 'Unknown' and is not a NetApp/N-Series 'CIFS' volume.
2017-02-08 11:57:13 ANS2832E Incremental by snapshot difference failed for \\nassrv6789\filevolume3$. See the error log for details.
2017-02-08 11:57:14 ANS1228E Sending of object '\\nassrv6789\filevolume3$' failed.
2017-02-08 11:57:14 ANS5283E The operation was unsuccessful.

Also tried to run this on the command line directly:

tsm> inc "\\nassrv6789\filevolume3$" -snapdiff

Incremental by snapshot difference of volume '\\nassrv6789\filevolume3$'
ANS2831E Incremental by snapshot difference cannot be performed on '\\nassrv6789\filevolume3$' as it is of type
'Unknown' and is not a NetApp/N-Series 'CIFS' volume.
ANS2832E Incremental by snapshot difference failed for \\nassrv6789\filevolume3$. See the error log for details.

ANS1228E Sending of object '\\nassrv6789\filevolume3$' failed.
ANS5283E The operation was unsuccessful.

tsm>
tsm> inc "\\nassrv6789\filevolume3$" -snapdiff -snapdiff -useexistingbase -basesnapshotname="yearly"

Incremental by snapshot difference of volume '\\nassrv6789\filevolume3$'
ANS2831E Incremental by snapshot difference cannot be performed on '\\nassrv6789\filevolume3$' as it is of type
'Unknown' and is not a NetApp/N-Series 'CIFS' volume.
ANS2832E Incremental by snapshot difference failed for \\nassrv6789\filevolume3$. See the error log for details.

ANS1228E Sending of object '\\nassrv6789\filevolume3$' failed.
ANS5283E The operation was unsuccessful.

tsm>

This is the first time that I try to run CIFS/snapdiff backup so is there anything I need to do on the netapp filer before this kind of backup is possible? I have added the domain user I run my backups as in the BUILTIN\administrators group and the BUILTIN\backup operators in the SVM (storage virtual machine) in my netapp filer.

/C
 
PREDATAR Control23

Unfortunately did this not help:

2017-02-08 11:57:13 ANS2831E Incremental by snapshot difference cannot be performed on '\\nassrv6789\filevolume3$' as it is of type 'Unknown' and is not a NetApp/N-Series 'CIFS' volume.
It did help, at least now the syntax is good, so one problem resolved.

Now you have to deal with this:
ANS2831E Incremental by snapshot difference cannot be performed on '\\nassrv6789\filevolume3$' as it is of type 'Unknown' and is not a NetApp/N-Series 'CIFS' volume.
The first 6 Google hits could potentially apply to you:
https://www.google.ca/search?q=ANS2831E
 
PREDATAR Control23

Will analyze if I got any better performance tomorrow! :)
 
PREDATAR Control23

Hi again!
Didn't get any better performance with -snapdiff
One thing that might be a problem for me is that I am running against a secondary cluster where all the volumes are readonly. I yesterday had to use '-snapdiff -useexistingbase -basesnapshotname="yearly"' to be able to read from the filesystem. When running a second backup with the exactly same command and snapshot, tsm seems to make a normal cifs compare. No difference as doing a normal cifs backup as far as I can see.

Is it possible to do any other tuning to speed up things? Right now I have ~15 MB/s transfer rate on a 10 Gbit interface, and the filer and tsm server is on the same subnet (no routing).
 
PREDATAR Control23

Not to sound rude but going back to what I said earlier on, I really stay away from NDMP backups. I never found a tweak to make things run faster even if the TSM Server was dedicated to it, running on the same subnet and had binded NICs to run at 20 GB. Speed was always at around 20 to 25 MB/sec near to what you are experiencing.

A 10 TiB CIFS NDMP FULL backup ran for almost four days and most of the time aborts. I hardly get completed backups. The short of it, we abandoned NDMP in favor of snapshots running 4 hourly and 30 dailies.
 
PREDATAR Control23

Not to sound rude but going back to what I said earlier on, I really stay away from NDMP backups. I never found a tweak to make things run faster even if the TSM Server was dedicated to it, running on the same subnet and had binded NICs to run at 20 GB. Speed was always at around 20 to 25 MB/sec near to what you are experiencing.

A 10 TiB CIFS NDMP FULL backup ran for almost four days and most of the time aborts. I hardly get completed backups. The short of it, we abandoned NDMP in favor of snapshots running 4 hourly and 30 dailies.

Yes.... I would love to stay away from Netapp and NDMP + TSM if possible...

My life is a little harder. My organizations demands are:
Primary netapp snapshot :
one every second hour saved in 48 hrs
weekend saved for 5 weeks
Secondary netapp snapshot:
one nightly saved for 90 days
TSM tape backups:
once a quarter a netapp secondary snapshot saved for 25 yrs

The only choice I found out that work with a proper speed is NDMP.
When running NDMP backups in CAB or node-scoped mode I get a transfer rate between ~100 - 300 MB/s which is good, but after 25 yrs we might not have a netapp to restore our data to cause of the NDMP t=NetApp Dump format.

If I don't solve this and find another way of backing up the netapp data into TSM I need to look for another way of storing my data in 25 yrs.

Right now I started to look into a NAS object store solution.
 
PREDATAR Control23

Right now I started to look into a NAS object store solution.

That will be a better alternative. It will be even better if you do an open solution that is hardware agnostic so you can mix backend storage without being confined by proprietary configurations.

On another note, I frown at retention periods that puts backup on electronic media for more than 10 years. As you pointed out, would the restore platform be even there after all those years?

DLT and DAT tape was the in-thing about 14 or so years ago. If you have set the retention for 25 years, do you think you can recover data now? Possibly yes if you have kept that DLT or DAT drive!

For a truly readable backup with retention of 25 or more years nothing beats a media that does not rely on any advanced electronics or sophisticated systems. Unfortunately, the IT and Business world has turned away from this simple backup system saying 'it is dated'.

Can anyone care to venture what this media is?
 
Last edited:
PREDATAR Control23

with snapdiff I have more than 26 volumes and windows is limited to 26 network drive. How do you do ? if anyone has a sample script who uses unc i would be happy :)
 
Top