• Please help support our sponsors by considering their products and services.
    Our sponsors enable us to serve you with this high-speed Internet connection and fast webservers you are currently using at ADSM.ORG.
    They support this free flow of information and knowledge exchange service at no cost to you.

    Please welcome our latest sponsor Tectrade . We can show our appreciation by learning more about Tectrade Solutions
  • Community Tip: Please Give Thanks to Those Sharing Their Knowledge.

    If you receive helpful answer on this forum, please show thanks to the poster by clicking "LIKE" link for the answer that you found helpful.

  • Community Tip: Forum Rules (PLEASE CLICK HERE TO READ BEFORE POSTING)

    Click the link above to access ADSM.ORG Acceptable Use Policy and forum rules which should be observed when using this website. Violators may be banned from this website. This notice will disappear after you have made at least 3 posts.

Best way to setup CIFS (unc) backup of a NAS via TSM?

combato

ADSM.ORG Member
#1
Hi Pro's!

I have some data on a NAS filer (netapp) where I need to save the backups for 25 years. Today I have solved this with NDMP backup and datamovers.
After I have spoken to some guys I have Heard that if I need to restore the data in 25 years from now I need to have a netapp system!! (Is this true??)
This made me worried.

I therefore have tried to backup my filer via CIFS (unc paths). The problem is that I have ~100 TB of data, located on ~50 volumes. How can I setup this in the best way?

One node for each volume share as I schedule with the objects option in a client schedule? (not so funny with fifty nodes)
One node for all volumes (yes please) with many schedules and several volumes specified in the client schedule via objects option?

Any other way of setting this up with the best performance and less administration?

/C
 

moon-buddy

ADSM.ORG Moderator
#2
If the intent is to restore data to its original state, then YES you need a Netapp to do that.

This is why I stay away from NDMP and Netapp (if I can).

If I were you, I would dump files on a disk and backup normally.
 

combato

ADSM.ORG Member
#3
Yes, the thing now is to setup the CIFS backup in the best way. Have so far tried to backup all shares volumes via one single node but it takes ages.

How can I restore NDMP data taken from a Netapp to Another location and/or system?
 

moon-buddy

ADSM.ORG Moderator
#4
Short and easy answer: you need a Netapp device.

Long way: backup the way I posted above and you will never need a Netapp device. Again, I stay away from NDMP and Netapp!
 

combato

ADSM.ORG Member
#5
Ok, thanks!
My problem is that all our storage systems worldwide are based on netapp technology so I have no choice.

I'll continue to run my CIFS backup and try to tune my network and TSM server so I can receive data faster. Right now I have an average network speed at 15 MB/s on a 1 Gbps connection. :(
 

fhignz

ADSM.ORG Member
#6
Hello,
to me it makes not such a big difference specifying 50 nodes and 50 schedules than to specifying only one node and 50 schedules. Perhaps scripting may help creating the commands and then "macro" the resulting command file.
I do netapp cifs backups, ~500 TB now with ~20 volumes in 20 nodes as agents to one proxynode target and 20 schedules using snapdiff (netapp snapshots for volumes have to be enabled). Sometimes snapshot processing fails in various volumes. For these a so called "full incremental" backup will be started which creates a new base snapshot so the next backup is a snapdiff again. In a full inc the TSM client API does the process of finding differences. In snapdiff the netapp will do. Backups are incrementals forever unlike NDMP where backups are fulls and diffs. Finding differences in 100 mio files takes about 15 to 20 hours in TSM, but just around 10 minutes with snapdiff.
In schedule's options a put: -snapdiff (for normal snapdiff processing) or -snapdiff -createnewbase (for a full inc, put in a onetime scheduler which will be activated in case of trouble). I also have an -asnodename - Option since I use PROXYNODE...
In the objects-field I put the object to backup up. E.g. \\nodename\unc_name
Probably one should perform "full incs" on a regular basis, since snapshot processing not always knows about all differences the TSM client would find...
Have a nice day,
Friedrich
 

combato

ADSM.ORG Member
#7
In schedule's options a put: -snapdiff (for normal snapdiff processing) or -snapdiff -createnewbase (for a full inc, put in a onetime scheduler which will be activated in case of trouble). I also have an -asnodename - Option since I use PROXYNODE...
In the objects-field I put the object to backup up. E.g. \\nodename\unc_name
Probably one should perform "full incs" on a regular basis, since snapshot processing not always knows about all differences the TSM client would find...
Have a nice day,
Friedrich
Hi Fredrich!
Thank you for your answer. I have been off the TSM work for some time (long time), but now started up where I left off and the same problem exists. I today tried to run a client scheduled backup with the following options:
tsm: TSMSERV1>q sch 1_YEARS CIFS_TEST_SCH f=d

Policy Domain Name: 10_YEARS
Schedule Name: CIFS_TEST_SCH
Description:
Action: Incremental
Subaction:
Options: -snapdiff
Objects: "\\nassrv6789\filevolume3$\"
Priority: 5
Start Date/Time: 09/09/2010 10:33:00
Duration: 1 Minute(s)
Maximum Run Time (Minutes): 0
Schedule Style: Enhanced
Period:
Day of Week: Any
Month: Feb
Day of Month: 8
Week of Month: Any
Expiration:
Last Update by (administrator): admin
Last Update Date/Time: 02/08/2017 10:32:46
Managing profile:

tsm: TSMSERV1>

I have a windows 2012 server as I use as node that I have assigned to the above schedule.
The error I get is:

2017-02-08 10:33:59 ------------------------------------------------------------
2017-02-08 10:33:59 Schedule Name: CIFS_TEST_SCH
2017-02-08 10:33:59 Action: Incremental
2017-02-08 10:33:59 Objects: "\\nassrv6789\filevolume3$\"
2017-02-08 10:33:59 Options: -snapdiff
2017-02-08 10:33:59 Server Window Start: 10:33:00 on 2017-02-08
2017-02-08 10:33:59 ------------------------------------------------------------
2017-02-08 10:33:59
Executing scheduled command now.
2017-02-08 10:34:01 --- SCHEDULEREC OBJECT BEGIN CIFS_TEST_SCH 2017-02-08 10:33:00
2017-02-08 10:34:01 ANS1670E The file specification is not valid. Specify a valid Network Appliance or N-Series NFS (AIX, Linux) or CIFS (Windows) volume.
2017-02-08 10:34:02 --- SCHEDULEREC STATUS BEGIN
2017-02-08 10:34:02 ANS5283E The operation was unsuccessful.

What might be wrong???

BR /C
 

combato

ADSM.ORG Member
#9
The trailing backslash
Unfortunately did this not help:

2017-02-08 11:57:13 ANS2831E Incremental by snapshot difference cannot be performed on '\\nassrv6789\filevolume3$' as it is of type 'Unknown' and is not a NetApp/N-Series 'CIFS' volume.
2017-02-08 11:57:13 ANS2832E Incremental by snapshot difference failed for \\nassrv6789\filevolume3$. See the error log for details.
2017-02-08 11:57:14 ANS1228E Sending of object '\\nassrv6789\filevolume3$' failed.
2017-02-08 11:57:14 ANS5283E The operation was unsuccessful.

Also tried to run this on the command line directly:

tsm> inc "\\nassrv6789\filevolume3$" -snapdiff

Incremental by snapshot difference of volume '\\nassrv6789\filevolume3$'
ANS2831E Incremental by snapshot difference cannot be performed on '\\nassrv6789\filevolume3$' as it is of type
'Unknown' and is not a NetApp/N-Series 'CIFS' volume.
ANS2832E Incremental by snapshot difference failed for \\nassrv6789\filevolume3$. See the error log for details.

ANS1228E Sending of object '\\nassrv6789\filevolume3$' failed.
ANS5283E The operation was unsuccessful.

tsm>
tsm> inc "\\nassrv6789\filevolume3$" -snapdiff -snapdiff -useexistingbase -basesnapshotname="yearly"

Incremental by snapshot difference of volume '\\nassrv6789\filevolume3$'
ANS2831E Incremental by snapshot difference cannot be performed on '\\nassrv6789\filevolume3$' as it is of type
'Unknown' and is not a NetApp/N-Series 'CIFS' volume.
ANS2832E Incremental by snapshot difference failed for \\nassrv6789\filevolume3$. See the error log for details.

ANS1228E Sending of object '\\nassrv6789\filevolume3$' failed.
ANS5283E The operation was unsuccessful.

tsm>

This is the first time that I try to run CIFS/snapdiff backup so is there anything I need to do on the netapp filer before this kind of backup is possible? I have added the domain user I run my backups as in the BUILTIN\administrators group and the BUILTIN\backup operators in the SVM (storage virtual machine) in my netapp filer.

/C
 

marclant

ADSM.ORG Moderator
#10
Unfortunately did this not help:

2017-02-08 11:57:13 ANS2831E Incremental by snapshot difference cannot be performed on '\\nassrv6789\filevolume3$' as it is of type 'Unknown' and is not a NetApp/N-Series 'CIFS' volume.
It did help, at least now the syntax is good, so one problem resolved.

Now you have to deal with this:
ANS2831E Incremental by snapshot difference cannot be performed on '\\nassrv6789\filevolume3$' as it is of type 'Unknown' and is not a NetApp/N-Series 'CIFS' volume.
The first 6 Google hits could potentially apply to you:
https://www.google.ca/search?q=ANS2831E
 

combato

ADSM.ORG Member
#11

combato

ADSM.ORG Member
#13
Hi again!
Didn't get any better performance with -snapdiff
One thing that might be a problem for me is that I am running against a secondary cluster where all the volumes are readonly. I yesterday had to use '-snapdiff -useexistingbase -basesnapshotname="yearly"' to be able to read from the filesystem. When running a second backup with the exactly same command and snapshot, tsm seems to make a normal cifs compare. No difference as doing a normal cifs backup as far as I can see.

Is it possible to do any other tuning to speed up things? Right now I have ~15 MB/s transfer rate on a 10 Gbit interface, and the filer and tsm server is on the same subnet (no routing).
 

moon-buddy

ADSM.ORG Moderator
#14
Not to sound rude but going back to what I said earlier on, I really stay away from NDMP backups. I never found a tweak to make things run faster even if the TSM Server was dedicated to it, running on the same subnet and had binded NICs to run at 20 GB. Speed was always at around 20 to 25 MB/sec near to what you are experiencing.

A 10 TiB CIFS NDMP FULL backup ran for almost four days and most of the time aborts. I hardly get completed backups. The short of it, we abandoned NDMP in favor of snapshots running 4 hourly and 30 dailies.
 

combato

ADSM.ORG Member
#15
Not to sound rude but going back to what I said earlier on, I really stay away from NDMP backups. I never found a tweak to make things run faster even if the TSM Server was dedicated to it, running on the same subnet and had binded NICs to run at 20 GB. Speed was always at around 20 to 25 MB/sec near to what you are experiencing.

A 10 TiB CIFS NDMP FULL backup ran for almost four days and most of the time aborts. I hardly get completed backups. The short of it, we abandoned NDMP in favor of snapshots running 4 hourly and 30 dailies.
Yes.... I would love to stay away from Netapp and NDMP + TSM if possible...

My life is a little harder. My organizations demands are:
Primary netapp snapshot :
one every second hour saved in 48 hrs
weekend saved for 5 weeks
Secondary netapp snapshot:
one nightly saved for 90 days
TSM tape backups:
once a quarter a netapp secondary snapshot saved for 25 yrs

The only choice I found out that work with a proper speed is NDMP.
When running NDMP backups in CAB or node-scoped mode I get a transfer rate between ~100 - 300 MB/s which is good, but after 25 yrs we might not have a netapp to restore our data to cause of the NDMP t=NetApp Dump format.

If I don't solve this and find another way of backing up the netapp data into TSM I need to look for another way of storing my data in 25 yrs.

Right now I started to look into a NAS object store solution.
 

moon-buddy

ADSM.ORG Moderator
#16
Right now I started to look into a NAS object store solution.
That will be a better alternative. It will be even better if you do an open solution that is hardware agnostic so you can mix backend storage without being confined by proprietary configurations.

On another note, I frown at retention periods that puts backup on electronic media for more than 10 years. As you pointed out, would the restore platform be even there after all those years?

DLT and DAT tape was the in-thing about 14 or so years ago. If you have set the retention for 25 years, do you think you can recover data now? Possibly yes if you have kept that DLT or DAT drive!

For a truly readable backup with retention of 25 or more years nothing beats a media that does not rely on any advanced electronics or sophisticated systems. Unfortunately, the IT and Business world has turned away from this simple backup system saying 'it is dated'.

Can anyone care to venture what this media is?
 
Last edited:

tpesselier

ADSM.ORG Member
#17
with snapdiff I have more than 26 volumes and windows is limited to 26 network drive. How do you do ? if anyone has a sample script who uses unc i would be happy :)
 

Advertise at ADSM.ORG

If you are reading this, so are your potential customer. Advertise at ADSM.ORG right now.

UpCloud high performance VPS at $5/month

Get started with $25 in credits on Cloud Servers. You must use link below to receive the credit. Use the promo to get upto 5 month of FREE Linux VPS.

The Spectrum Protect TLA (Three-Letter Acronym): ISP or something else?

  • Every product needs a TLA, Let's call it ISP (IBM Spectrum Protect).

    Votes: 12 16.0%
  • Keep using TSM for Spectrum Protect.

    Votes: 48 64.0%
  • Let's be formal and just say Spectrum Protect

    Votes: 9 12.0%
  • Other (please comement)

    Votes: 6 8.0%

Forum statistics

Threads
31,321
Messages
133,392
Members
21,453
Latest member
RRhodes
Top