• Please help support our sponsors by considering their products and services.
    Our sponsors enable us to serve you with this high-speed Internet connection and fast webservers you are currently using at ADSM.ORG.
    They support this free flow of information and knowledge exchange service at no cost to you.

    Please welcome our latest sponsor Tectrade . We can show our appreciation by learning more about Tectrade Solutions
  • Community Tip: Please Give Thanks to Those Sharing Their Knowledge.

    If you receive helpful answer on this forum, please show thanks to the poster by clicking "LIKE" link for the answer that you found helpful.

  • Community Tip: Forum Rules (PLEASE CLICK HERE TO READ BEFORE POSTING)

    Click the link above to access ADSM.ORG Acceptable Use Policy and forum rules which should be observed when using this website. Violators may be banned from this website. This notice will disappear after you have made at least 3 posts.

How to migrate only premigrated files using GPFS driven policy?

#1
Hi all,

in order to avoid new migration of resident files, I would like to re-migrate the files that are in premigrated status modifying the GPFS policy that is managing HSM data migration.

I created a new policy file and I tested it using "mmapplypolicy -I test" command.
No errors have been found in the policy configuration, but it seems that the premigrated files in "sysm02" directory are not considered from policy.

Could you please help me to understand what is wrong?
Thank you in advance,
Mauro

premigrated files in "sysm02" directory:

dsmls /archive/sysm02/TEST_FILES/*
/archive/sysm02/TEST_FILES:
2147483648 2147483648 2097152 p test.02.2G
2147483648 2147483648 2097152 p test.03.2G

output of mmapplypolicy test:

mmapplypolicy /dev/gpfs_archive -I test -P athena-lowspace.onlypremigrated.2019.03.21
GPFS Current Data Pool Utilization in KB and %
system 159811014656 183542743040 87.070190%
3145665 of 134217728 inodes used: 2.343703%.
Loaded policy rules from athena-lowspace.onlypremigrated.2019.03.21.
Evaluating MIGRATE/DELETE/EXCLUDE rules with CURRENT_TIMESTAMP = [email protected]:00:32 UTC
parsed 1 Placement Rules, 0 Restore Rules, 3 Migrate/Delete/Exclude Rules,
0 List Rules, 1 External Pool/List Rules





RULE EXTERNAL POOL 'hsm' EXEC '/var/mmfs/etc/mmpolicyExec-hsm.athena' OPTS '-v'
RULE 'exclude hsm system files' EXCLUDE WHERE PATH_NAME LIKE '%/.SpaceMan%'
RULE 'exclude shared directory' EXCLUDE WHERE PATH_NAME LIKE '%/shared%'
/* RULE 'exclude bad users dir' EXCLUDE WHERE exclude_list */


RULE 'hsmdata' MIGRATE FROM POOL 'system' THRESHOLD(85,84) WEIGHT(CURRENT_TIMESTAMP - ACCESS_TIME) TO POOL 'hsm' WHERE ( ((
PATH_NAME LIKE '%/sysm02%'
)
) AND ((
MISC_ATTRIBUTES LIKE '%M%' AND MISC_ATTRIBUTES NOT LIKE '%V%'
)
) AND ( FILE_SIZE >= 943718400 ) AND ( FILE_SIZE <= 322122547200 ) AND (NAME LIKE '%.nc' OR NAME LIKE '%.gz' OR NAME LIKE '%.tar' OR NAME LIKE '%.bz' OR NAME LIKE '%.zip' OR NAME LIKE '%.7z' OR NAME LIKE '%.taz' OR NAME LIKE '%.tbz' OR NAME LIKE '%.tgz' OR NAME LIKE '%.tz' OR NAME LIKE '%.z') )

RULE 'Default' SET POOL 'system'
/var/mmfs/etc/mmpolicyExec-hsm.athena TEST /archive -v
TEST -x /usr/bin/dsmc
TEST -x /usr/bin/dsmmigrate
TEST -x /usr/bin/dsmrecall
/var/mmfs/etc/mmpolicyExec-hsm.athena: TEST Ok
[email protected]:02:12.572 Directory entries scanned: 3141148.
Directories scan: 2903555 files, 130011 directories, 107582 other objects, 0 'skipped' files and/or errors.
[email protected]:02:17.873 Sorting 3141148 file list records.
Inodes scan: 2903554 files, 130011 directories, 107583 other objects, 1 'skipped' files and/or errors.
[email protected]:02:43.251 Policy evaluation. 3141148 files scanned.
[email protected]:02:43.257 Sorting 0 candidate file list records.
[email protected]:02:43.257 Choosing candidate files. 0 records scanned.
Summary of Rule Applicability and File Choices:
Rule# Hit_Cnt KB_Hit Chosen KB_Chosen KB_Ill Rule
0 373 618427520 0 0 0 RULE 'exclude hsm system files' EXCLUDE WHERE(.)
1 0 0 0 0 0 RULE 'exclude shared directory' EXCLUDE WHERE(.)
2 0 0 0 0 0 RULE 'hsmdata' MIGRATE FROM POOL 'system' THRESHOLD(85,84) WEIGHT(.) TO POOL 'hsm' WHERE(.)

Filesystem objects with no applicable rules: 3140774.

GPFS Policy Decisions and File Choice Totals:
Chose to migrate 0KB: 0 of 0 candidates;
Chose to premigrate 0KB: 0 candidates;
Already co-managed 0KB: 0 candidates;
Chose to delete 0KB: 0 of 0 candidates;
Chose to list 0KB: 0 of 0 candidates;
Chose to change encryption 0KB: 0 of 0 candidates;
0KB of chosen data is illplaced or illreplicated;
Predicted Data Pool Utilization in KB and %:
system 159818178560 183542743040 87.074093%

this is the policy file:

**************************************************
define(
selected_list,
(
PATH_NAME LIKE '%/sysm02%'
)
)

define(
is_premigrated,
(
MISC_ATTRIBUTES LIKE '%M%' AND MISC_ATTRIBUTES NOT LIKE '%V%'
)
)

RULE EXTERNAL POOL 'hsm' EXEC '/var/mmfs/etc/mmpolicyExec-hsm.athena' OPTS '-v'
RULE 'exclude hsm system files' EXCLUDE WHERE PATH_NAME LIKE '%/.SpaceMan%'
RULE 'exclude shared directory' EXCLUDE WHERE PATH_NAME LIKE '%/shared%'

RULE 'hsmdata' MIGRATE FROM POOL 'system' THRESHOLD(85,84) WEIGHT(CURRENT_TIMESTAMP - ACCESS_TIME) TO POOL 'hsm' WHERE ( (selected_list) AND (is_premigrated) AND ( FILE_SIZE >= 943718400 ) AND ( FILE_SIZE <= 322122547200 ) AND (NAME LIKE '%.nc' OR NAME LIKE '%.gz' OR NAME LIKE '%.tar' OR NAME LIKE '%.bz' OR NAME LIKE '%.zip' OR NAME LIKE '%.7z' OR NAME LIKE '%.taz' OR NAME LIKE '%.tbz' OR NAME LIKE '%.tgz' OR NAME LIKE '%.tz' OR NAME LIKE '%.z') )

RULE 'Default' SET POOL 'system'

**************************************************
 

Advertise at ADSM.ORG

If you are reading this, so are your potential customer. Advertise at ADSM.ORG right now.

UpCloud high performance VPS at $5/month

Get started with $25 in credits on Cloud Servers. You must use link below to receive the credit. Use the promo to get upto 5 month of FREE Linux VPS.

The Spectrum Protect TLA (Three-Letter Acronym): ISP or something else?

  • Every product needs a TLA, Let's call it ISP (IBM Spectrum Protect).

    Votes: 16 18.8%
  • Keep using TSM for Spectrum Protect.

    Votes: 52 61.2%
  • Let's be formal and just say Spectrum Protect

    Votes: 10 11.8%
  • Other (please comement)

    Votes: 7 8.2%

Forum statistics

Threads
31,448
Messages
133,974
Members
21,549
Latest member
idhelmyy
Top