Results 1 to 4 of 4
  1. #1
    Newcomer
    Join Date
    Jan 2012
    Posts
    2
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default TSM with NetApp data deduplication - ASIS

    Hi There

    I have a requirement to store 15-30 days of active data on disks for quick restore. Just want to get feedback from community if anyone has tested TSM against NetApp data deduplication technology (Advance Single Instance Storage) and if yes
    1) type of data backed up
    2) how much data backed up daily and
    3) what sort of data reduction factor they are getting. Thanks in advance.

    Regards
    TSMmer

  2. #2
    Moderator Harry_Redl's Avatar
    Join Date
    Dec 2003
    Location
    Czech Republic
    Posts
    2,220
    Thanks
    3
    Thanked 100 Times in 99 Posts

    Default

    Hi,

    this is how it works - NetApp ASIS is INTERNAL thing only - so all the applications see the data in FULL size. Whenever you read files, you got them in full. NDMP does read the files - so even NDMP backups read the full size.
    The only method that benefits from the deduplication is the SnapmirrorToTape - subclass of NDMP backups - as it is block-level transfer encapsulated in NDMP protocol.

    Hope following example says it all:

    single volume with deduplication enabled - I have put one 150MB file in two folders and run the "sis start" - this is the result:
    Code:
    netapp1> df -h dedup
    Filesystem               total       used      avail capacity  Mounted on
    /vol/dedup/              475MB      151MB      323MB      32%  /vol/dedup/
    /vol/dedup/.snapshot       25MB       84KB       24MB       0%  /vol/dedup/.snapshot
    
    netapp1> df -sh dedup
    Filesystem                used      saved       %saved
    /vol/dedup/              151MB      150MB          50%
    netapp1>


    This is how "normal" NDMP works - you can see it transfers cca 300MB:
    Code:
    tsm: xxxx>ba node simulator /vol/dedup mode=full toc=no
    
    Date/Time                Message
    --------------------     ----------------------------------------------------------
    01/03/2012 13:09:00      ANR0984I Process 88 for BACKUP NAS (FULL) started in the BACKGROUND at 01:09:00 PM. (SESSION: 6359, PROCESS: 88)
    01/03/2012 13:09:00      ANR1063I Full backup of NAS node SIMULATOR, file system /vol/dedup, started as process 88 
    by administrator ADMIN. (SESSION: 6359, PROCESS: 88)
    01/03/2012 13:15:14      ANR1067I NAS Backup to TSM Storage process 88 completed. (SESSION: 6359, PROCESS: 88)
    01/03/2012 13:15:14      ANR0986I Process 88 for BACKUP NAS (FULL) running in the BACKGROUND processed 1 items for 
    a total of 316,436,480 bytes with a completion state of SUCCESS at 01:15:14 PM. (SESSION: 6359, PROCESS: 88)
    01/03/2012 13:24:09      ANR2017I Administrator ADMIN issued command: QUERY ACTLOG search='process 88'  (SESSION: 6359)
    and this is how SMtT works - transfers about 160MB (few MB of metadata ....)

    Code:
    tsm: xxxx>ba node simulator /vol/dedup mode=full type=snapmirror
    
    Date/Time                Message
    --------------------     ----------------------------------------------------------
    01/03/2012 13:19:37      ANR0984I Process 89 for NAS SNAPMIRROR BACKUP started in the BACKGROUND at 01:19:37 PM. 
    (SESSION: 6359, PROCESS: 89)
    01/03/2012 13:19:37      ANR2685I SnapMirror backup of NAS node SIMULATOR, file system /vol/dedup, started as process 
    89 by administrator ADMIN. (SESSION: 6359, PROCESS: 89)
    01/03/2012 13:21:06      ANR1067I NAS SnapMirror Backup process 89 completed. (SESSION: 6359, PROCESS: 89)
    01/03/2012 13:21:06      ANR0986I Process 89 for NAS SNAPMIRROR BACKUP running in the BACKGROUND processed 1 items
     for a total of 161,161,216 bytes with a completion state of SUCCESS at 01:21:06 PM. (SESSION: 6359, PROCESS: 89)
    01/03/2012 13:24:20      ANR2017I Administrator ADMIN issued command: QUERY ACTLOG search='process 89'  (SESSION: 6359)

    Using SMtT does not allow incrementals and single file restore.

    Hope it helps

    Harry
    Last edited by Harry_Redl; 01-03-2012 at 05:47 PM.

  3. #3
    Newcomer
    Join Date
    Jan 2012
    Posts
    2
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default

    Harry - Thanks for your comments. I am however keen to get feedback from community on
    1) How much data storage reduction have been achieved through the use of NetApp ASIS e.g. when backing up 10TB/day and keeping data (TSM storage pool) on disks for say 10,20 or 30 days?
    2) What were the issues if any

    With 10TB/day over 30 days TSM storage pool on NetApp will be 300TB. With ASIS dedup enabled depending on the reduction factor the size will be smaller.

    This is really a data deduplication related question as other options would be to use 1) TSM client/server deduplication 2) an in-line data deduplication tool from EMC or IBM 3) Data compression hardware like IBM RTC

    Regards
    TSMmer

  4. #4
    Moderator Harry_Redl's Avatar
    Join Date
    Dec 2003
    Location
    Czech Republic
    Posts
    2,220
    Thanks
    3
    Thanked 100 Times in 99 Posts

    Default

    Hi,

    sorry - I misunderstood the question.
    Have no numbers here but few pointers.
    a) Data ONTAP lower than 8.1 has a limit for volume size to be deduplicted - it used to be platform dependent with maximum of 16TB. With ONTAP 8.1 the limit is the maximum volume size you can create (which is still platform dependant)
    So even with high-end filer (which I assume you have better usage for than using it for backup) you have to create such a large pool from multiple NetApp volumes - and the NetApp deduplication is done on volume level
    b) NetApp deduplication works better with NAS approach - as it really frees the size in the volume (and you see the freed space immediately). When you use the ASIS on the block device (LUN) - the space is freed as well - but the filesystem on the LUN is not aware of that. So it is a bit more complicated to use this space.
    c) reclamation of file volumes makes the deduplication ratio worse as for NetApp filesystem the blocks change ....


    Harry

Similar Threads

  1. Replies: 1
    Last Post: 10-27-2011, 04:53 PM
  2. Using NetApp Deduplication on a Dedup Pool
    By mclawler in forum TSM Operation
    Replies: 3
    Last Post: 06-08-2011, 02:51 PM
  3. Replies: 22
    Last Post: 11-06-2010, 10:43 PM
  4. TSM Data Deduplication
    By pfsubaru in forum TSM Operation
    Replies: 7
    Last Post: 04-01-2010, 03:25 AM
  5. TSM & Data Deduplication
    By influx in forum Others
    Replies: 3
    Last Post: 03-12-2008, 05:24 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •