ADSM-L

[ADSM-L] How to backup Billions of files ? Use mmbackup together with Spectrum Protect

2017-03-20 09:07:31
Subject: [ADSM-L] How to backup Billions of files ? Use mmbackup together with Spectrum Protect
From: Frank Kraemer <kraemerf AT DE.IBM DOT COM>
To: ADSM-L AT VM.MARIST DOT EDU
Date: Mon, 20 Mar 2017 14:00:21 +0100
Bo,
this is easy if you just use the right filesystem to hold the data :-)
We have multiple very large scale customers using IBM Spectrum Scale for
this.

Environment: Medium IBM Spectrum Scale installations with a single digit
number of nodes and file systems. IBM Spectrum Protect backup archive
client installed on several cluster nodes

• Scalability: Tens of millions of files, Tens of terrabytes of data, up to
1.000.000.000 Objects (empirical value)
• Processing: IBM Spectrum Scale mmbackup scans file system and IBM
Spectrum Protect data base and generates list of backup candidates.
IBM Spectrum Protect backup archive client used from mmbackup to perform
file system backup.
• Pros: Simple setup and usage, Good performance and scalability
• Cons: All data goes to one IBM Spectrum Protect server

Backup cycle:
• After start mmbackup evaluates the cluster environment and verifies
product versions and settings
• Optional the Spectrum Protect server is queried for existing backup
information. In other cases existing shadow DB is used for processing
• The GPFS (Spectrum Scale) policy engine is used to generate a list files
currently eligible for backup activities
• Compare existing shadow DB and scan result to calculate file lists for
required backup activities
• Expire all files deleted in the file system since last backup run
• Incremental backup all files with changed metadata in the file system
since last backup run
• Selective backup all files with changed data in the file system since
last backup run
• While backup activities ongoing update shadow DB inline
• Analyse backup results from all used cluster nodes and finish backup
cycle by selective backup the current shadow DB

If this is not enough..see:
https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/Tivoli%20Storage%20Manager/page/Petascale%20Data%20Protection


-frank-

Frank Kraemer
IBM Consulting IT Specialist / Client Technical Architect
Am Weiher 24, 65451 Kelsterbach
mailto:kraemerf AT de.ibm DOT com
voice: +49-(0)171-3043699 / +4970342741078
IBM Germany
<Prev in Thread] Current Thread [Next in Thread>
  • [ADSM-L] How to backup Billions of files ? Use mmbackup together with Spectrum Protect, Frank Kraemer <=