HBA Drivers - EMLXS vs LPFC

[GregE]

I'm getting there. Actually, I think I'm there except for one thing that may not be an issue, but it appears to be. Mikey_D, thanks for your help. You once again got me going in the right direction on this. The documentation (to me anyway) isn't as straightforward as I would have expected.
__

Is anyone familiar enough to know if the path info listed from a "tapelist" is significant (Path Type / Current Active Path)? It's labeling all paths as the Alternate path, yet saying the active path is the Primary path (though none are labeled as Primary paths). The 100 series should be Primary, the 900 should be the Alternate. As for the library paths, the 521 should be Primary, the 526 should be the Alternate.

               ------------ IBM Tape Device Information ---------------
Inst#  Special File     Device           Serial No      TGT/LUN Ucode   World Wide NN    World Wide PN    Device Physical Path                         Path Type   Current Active Path
----- --------------- ---------------- ---------------- ------- ------ ---------------- ---------------- -------------------------------------------- ---------- -------------------
20    /dev/rmt/101st  ULT3580-TD4(e)   00078C0869       81664/0   A239   500507630F362901 500507630F762901 /devices/pci@3,700000/SUNW,emlxs@0,1/fp@0,0/IBMtape@w500507630f762901,0  Alt_path_1  Primary
20    /dev/rmt/901st  ULT3580-TD4(e)   00078C0869       81664/0   A239   500507630F362901 500507630F762901 /devices/pci@3,700000/SUNW,emlxs@0,1/fp@0,0/IBMtape@w500507630f762901,0  Alt_path_1  Primary
21    /dev/rmt/521smc 03584L32         0000078249470401 81664/1   A420         N/A             N/A        /devices/pci@3,700000/SUNW,emlxs@0,1/fp@0,0/IBMtape@w500507630f762901,1  Alt_path_1  Primary
22    /dev/rmt/102st  ULT3580-TD4(e)   00078C0870       81152/0   A239   500507630F362902 500507630F762902 /devices/pci@3,700000/SUNW,emlxs@0,1/fp@0,0/IBMtape@w500507630f762902,0  Alt_path_1  Primary
22    /dev/rmt/902st  ULT3580-TD4(e)   00078C0870       81152/0   A239   500507630F362902 500507630F762902 /devices/pci@3,700000/SUNW,emlxs@0,1/fp@0,0/IBMtape@w500507630f762902,0  Alt_path_1  Primary
23    /dev/rmt/103st  ULT3580-TD4(e)   00078C0865       80640/0   A239   500507630F362903 500507630F762903 /devices/pci@3,700000/SUNW,emlxs@0,1/fp@0,0/IBMtape@w500507630f762903,0  Alt_path_1  Primary
23    /dev/rmt/903st  ULT3580-TD4(e)   00078C0865       80640/0   A239   500507630F362903 500507630F762903 /devices/pci@3,700000/SUNW,emlxs@0,1/fp@0,0/IBMtape@w500507630f762903,0  Alt_path_1  Primary
24    /dev/rmt/104st  ULT3580-TD4(e)   00078C086A       80128/0   A239   500507630F362904 500507630F762904 /devices/pci@3,700000/SUNW,emlxs@0,1/fp@0,0/IBMtape@w500507630f762904,0  Alt_path_1  Primary
24    /dev/rmt/904st  ULT3580-TD4(e)   00078C086A       80128/0   A239   500507630F362904 500507630F762904 /devices/pci@3,700000/SUNW,emlxs@0,1/fp@0,0/IBMtape@w500507630f762904,0  Alt_path_1  Primary
26    /dev/rmt/105st  ULT3580-TD4(e)   00078C0880       81408/0   A239   500507630F362905 500507630F762905 /devices/pci@3,700000/SUNW,emlxs@0,1/fp@0,0/IBMtape@w500507630f762905,0  Alt_path_1  Primary
26    /dev/rmt/905st  ULT3580-TD4(e)   00078C0880       81408/0   A239   500507630F362905 500507630F762905 /devices/pci@3,700000/SUNW,emlxs@0,1/fp@0,0/IBMtape@w500507630f762905,0  Alt_path_1  Primary
29    /dev/rmt/106st  ULT3580-TD4(e)   00078C087D       80896/0   A239   500507630F362906 500507630F762906 /devices/pci@3,700000/SUNW,emlxs@0,1/fp@0,0/IBMtape@w500507630f762906,0  Alt_path_1  Primary
29    /dev/rmt/906st  ULT3580-TD4(e)   00078C087D       80896/0   A239   500507630F362906 500507630F762906 /devices/pci@3,700000/SUNW,emlxs@0,1/fp@0,0/IBMtape@w500507630f762906,0  Alt_path_1  Primary
43    /dev/rmt/526smc 03584L32         0000078249470401 80896/1   A420         N/A             N/A        /devices/pci@3,700000/SUNW,emlxs@0,1/fp@0,0/IBMtape@w500507630f762906,1  Alt_path_3  Primary
31    /dev/rmt/107st  ULT3580-TD4(e)   00078C087E       80384/0   A239   500507630F362907 500507630F762907 /devices/pci@3,700000/SUNW,emlxs@0,1/fp@0,0/IBMtape@w500507630f762907,0  Alt_path_1  Primary
31    /dev/rmt/907st  ULT3580-TD4(e)   00078C087E       80384/0   A239   500507630F362907 500507630F762907 /devices/pci@3,700000/SUNW,emlxs@0,1/fp@0,0/IBMtape@w500507630f762907,0  Alt_path_1  Primary
33    /dev/rmt/108st  ULT3580-TD4(e)   00078C0884       79872/0   A239   500507630F362908 500507630F762908 /devices/pci@3,700000/SUNW,emlxs@0,1/fp@0,0/IBMtape@w500507630f762908,0  Alt_path_1  Primary
33    /dev/rmt/908st  ULT3580-TD4(e)   00078C0884       79872/0   A239   500507630F362908 500507630F762908 /devices/pci@3,700000/SUNW,emlxs@0,1/fp@0,0/IBMtape@w500507630f762908,0  Alt_path_1  Primary
35    /dev/rmt/109st  ULT3580-TD4(e)   00078C0883       79616/0   A239   500507630F362909 500507630F762909 /devices/pci@3,700000/SUNW,emlxs@0,1/fp@0,0/IBMtape@w500507630f762909,0  Alt_path_1  Primary
35    /dev/rmt/909st  ULT3580-TD4(e)   00078C0883       79616/0   A239   500507630F362909 500507630F762909 /devices/pci@3,700000/SUNW,emlxs@0,1/fp@0,0/IBMtape@w500507630f762909,0  Alt_path_1  Primary
37    /dev/rmt/110st  ULT3580-TD4(e)   00078C0881       79104/0   A239   500507630F36290A 500507630F76290A /devices/pci@3,700000/SUNW,emlxs@0,1/fp@0,0/IBMtape@w500507630f76290a,0  Alt_path_1  Primary
37    /dev/rmt/910st  ULT3580-TD4(e)   00078C0881       79104/0   A239   500507630F36290A 500507630F76290A /devices/pci@3,700000/SUNW,emlxs@0,1/fp@0,0/IBMtape@w500507630f76290a,0  Alt_path_1  Primary

Here's my config.

.../etc/devlink.tab

# /dev/rmt/smc (medium changer, ie. tape library) #
#
type=ddi_byte:tape;addr=w500507630f762901,1;    rmt/521\M0
type=ddi_byte:tape;addr=w500507630f762906,1;    rmt/526\M0
#
# /dev/rmt/st (tape drives)  #
#
type=ddi_byte:tape;addr=w500507630f762901,0;    rmt/101\M0
type=ddi_byte:tape;addr=w500507630f762902,0;    rmt/102\M0
type=ddi_byte:tape;addr=w500507630f762903,0;    rmt/103\M0
type=ddi_byte:tape;addr=w500507630f762904,0;    rmt/104\M0
type=ddi_byte:tape;addr=w500507630f762905,0;    rmt/105\M0
type=ddi_byte:tape;addr=w500507630f762906,0;    rmt/106\M0
type=ddi_byte:tape;addr=w500507630f762907,0;    rmt/107\M0
type=ddi_byte:tape;addr=w500507630f762908,0;    rmt/108\M0
type=ddi_byte:tape;addr=w500507630f762909,0;    rmt/109\M0
type=ddi_byte:tape;addr=w500507630f76290a,0;    rmt/110\M0
#
## Alternate Path ##
#
# /dev/rmt/st (tape drives)  #
#
type=ddi_byte:tape;addr=w500507630f762901,0;    rmt/901\M0
type=ddi_byte:tape;addr=w500507630f762902,0;    rmt/902\M0
type=ddi_byte:tape;addr=w500507630f762903,0;    rmt/903\M0
type=ddi_byte:tape;addr=w500507630f762904,0;    rmt/904\M0
type=ddi_byte:tape;addr=w500507630f762905,0;    rmt/905\M0
type=ddi_byte:tape;addr=w500507630f762906,0;    rmt/906\M0
type=ddi_byte:tape;addr=w500507630f762907,0;    rmt/907\M0
type=ddi_byte:tape;addr=w500507630f762908,0;    rmt/908\M0
type=ddi_byte:tape;addr=w500507630f762909,0;    rmt/909\M0
type=ddi_byte:tape;addr=w500507630f76290a,0;    rmt/910\M0

.../usr/kernel/drv/IBMtape.conf

# Dual Path Support
#
dpf_support=1;

# fc-port-wwn=<wwpn> lun=0 (tape drive)
# fc-port-wwn=<wwpn> lun=1 (medium changer, ie. tape library)
#
name="IBMtape" parent="fp" fc-port-wwn="500507630f762901" lun=0;
name="IBMtape" parent="fp" fc-port-wwn="500507630f762901" lun=1;
name="IBMtape" parent="fp" fc-port-wwn="500507630f762902" lun=0;
name="IBMtape" parent="fp" fc-port-wwn="500507630f762903" lun=0;
name="IBMtape" parent="fp" fc-port-wwn="500507630f762904" lun=0;
name="IBMtape" parent="fp" fc-port-wwn="500507630f762905" lun=0;
name="IBMtape" parent="fp" fc-port-wwn="500507630f762906" lun=0;
name="IBMtape" parent="fp" fc-port-wwn="500507630f762906" lun=1;
name="IBMtape" parent="fp" fc-port-wwn="500507630f762907" lun=0;
name="IBMtape" parent="fp" fc-port-wwn="500507630f762908" lun=0;
name="IBMtape" parent="fp" fc-port-wwn="500507630f762909" lun=0;
name="IBMtape" parent="fp" fc-port-wwn="500507630f76290a" lun=0;

name="IBMtape" parent="pseudo" instance=16382;
name="IBMtape" parent="pseudo" instance=16383;

name="IBMtape" class="scsi" fc-port-wwn="500507630f762901" lun=0 block_size=0 buffering=1 immediate=0 trailer=0 sili=0;
name="IBMtape" class="scsi" fc-port-wwn="500507630f762901" lun=1 block_size=0 buffering=1 immediate=0 trailer=0 sili=0;
name="IBMtape" class="scsi" fc-port-wwn="500507630f762902" lun=0 block_size=0 buffering=1 immediate=0 trailer=0 sili=0;
name="IBMtape" class="scsi" fc-port-wwn="500507630f762903" lun=0 block_size=0 buffering=1 immediate=0 trailer=0 sili=0;
name="IBMtape" class="scsi" fc-port-wwn="500507630f762904" lun=0 block_size=0 buffering=1 immediate=0 trailer=0 sili=0;
name="IBMtape" class="scsi" fc-port-wwn="500507630f762905" lun=0 block_size=0 buffering=1 immediate=0 trailer=0 sili=0;
name="IBMtape" class="scsi" fc-port-wwn="500507630f762906" lun=0 block_size=0 buffering=1 immediate=0 trailer=0 sili=0;
name="IBMtape" class="scsi" fc-port-wwn="500507630f762906" lun=1 block_size=0 buffering=1 immediate=0 trailer=0 sili=0;
name="IBMtape" class="scsi" fc-port-wwn="500507630f762907" lun=0 block_size=0 buffering=1 immediate=0 trailer=0 sili=0;
name="IBMtape" class="scsi" fc-port-wwn="500507630f762908" lun=0 block_size=0 buffering=1 immediate=0 trailer=0 sili=0;
name="IBMtape" class="scsi" fc-port-wwn="500507630f762909" lun=0 block_size=0 buffering=1 immediate=0 trailer=0 sili=0;
name="IBMtape" class="scsi" fc-port-wwn="500507630f76290a" lun=0 block_size=0 buffering=1 immediate=0 trailer=0 sili=0;

 

[GregE]

I've been working with IBM support, and Sun support. Notice in my tapelist above, Physical Device Path is the same for every tape drive, with the same serial number on one HBA. It should be 10 serial numbers on one path, and 10 on another. If I pull out all of my tape info from /etc/devlink.tab (removing persistent bindings), notice the result is that tape drives are showing as attached to separate HBAs, which is correct.........

/opt/IBMtape/tapelist -A

         --------------- IBM Tape Device and HBA Information ---------------------
Inst#  Special File     Device           Serial No      TGT/LUN Ucode   World Wide NN    World Wide PN    Device Physical Path
----- --------------- ---------------- ---------------- ------- ------ ---------------- ---------------- --------------------------------------------
HBA Info - (Instance: 5  Driver: emlxs ) +++
20    /dev/rmt/12st   ULT3580-TD4      00078C0869       81664/0   A239   500507630F362901 500507630F762901  /devices/pci@3,700000/SUNW,emlxs@0,1/fp@0,0/IBMtape@w500507630f762901,0
22    /dev/rmt/14st   ULT3580-TD4      00078C0870       81152/0   A239   500507630F362902 500507630F762902  /devices/pci@3,700000/SUNW,emlxs@0,1/fp@0,0/IBMtape@w500507630f762902,0
23    /dev/rmt/15st   ULT3580-TD4      00078C0865       80640/0   A239   500507630F362903 500507630F762903  /devices/pci@3,700000/SUNW,emlxs@0,1/fp@0,0/IBMtape@w500507630f762903,0
21    /dev/rmt/13smc  03584L32         0000078249470401 81664/1   A420         N/A             N/A        /devices/pci@3,700000/SUNW,emlxs@0,1/fp@0,0/IBMtape@w500507630f762901,1
24    /dev/rmt/16st   ULT3580-TD4      00078C086A       80128/0   A239   500507630F362904 500507630F762904  /devices/pci@3,700000/SUNW,emlxs@0,1/fp@0,0/IBMtape@w500507630f762904,0
26    /dev/rmt/17st   ULT3580-TD4      00078C0880       81408/0   A239   500507630F362905 500507630F762905  /devices/pci@3,700000/SUNW,emlxs@0,1/fp@0,0/IBMtape@w500507630f762905,0
29    /dev/rmt/18st   ULT3580-TD4      00078C087D       80896/0   A239   500507630F362906 500507630F762906  /devices/pci@3,700000/SUNW,emlxs@0,1/fp@0,0/IBMtape@w500507630f762906,0
31    /dev/rmt/20st   ULT3580-TD4      00078C087E       80384/0   A239   500507630F362907 500507630F762907  /devices/pci@3,700000/SUNW,emlxs@0,1/fp@0,0/IBMtape@w500507630f762907,0
33    /dev/rmt/21st   ULT3580-TD4      00078C0884       79872/0   A239   500507630F362908 500507630F762908  /devices/pci@3,700000/SUNW,emlxs@0,1/fp@0,0/IBMtape@w500507630f762908,0
35    /dev/rmt/22st   ULT3580-TD4      00078C0883       79616/0   A239   500507630F362909 500507630F762909  /devices/pci@3,700000/SUNW,emlxs@0,1/fp@0,0/IBMtape@w500507630f762909,0
37    /dev/rmt/23st   ULT3580-TD4      00078C0881       79104/0   A239   500507630F36290A 500507630F76290A  /devices/pci@3,700000/SUNW,emlxs@0,1/fp@0,0/IBMtape@w500507630f76290a,0
43    /dev/rmt/19smc  03584L32         0000078249470401 80896/1   A420         N/A             N/A        /devices/pci@3,700000/SUNW,emlxs@0,1/fp@0,0/IBMtape@w500507630f762906,1

HBA Info - (Instance: 0  Driver: emlxs ) +++
25    /dev/rmt/0st    ULT3580-TD4      00078C0869       81664/0   A239   500507630F362901 500507630F762901  /devices/pci@0,600000/pci@0/pci@8/pci@0,1/SUNW,emlxs@1/fp@0,0/IBMtape@w500507630f762901,0
27    /dev/rmt/1smc   03584L32         0000078249470401 81664/1   A420         N/A             N/A        /devices/pci@0,600000/pci@0/pci@8/pci@0,1/SUNW,emlxs@1/fp@0,0/IBMtape@w500507630f762901,1
28    /dev/rmt/2st    ULT3580-TD4      00078C0870       81152/0   A239   500507630F362902 500507630F762902  /devices/pci@0,600000/pci@0/pci@8/pci@0,1/SUNW,emlxs@1/fp@0,0/IBMtape@w500507630f762902,0
30    /dev/rmt/3st    ULT3580-TD4      00078C0865       80640/0   A239   500507630F362903 500507630F762903  /devices/pci@0,600000/pci@0/pci@8/pci@0,1/SUNW,emlxs@1/fp@0,0/IBMtape@w500507630f762903,0
32    /dev/rmt/4st    ULT3580-TD4      00078C086A       80128/0   A239   500507630F362904 500507630F762904  /devices/pci@0,600000/pci@0/pci@8/pci@0,1/SUNW,emlxs@1/fp@0,0/IBMtape@w500507630f762904,0
34    /dev/rmt/5st    ULT3580-TD4      00078C0880       81408/0   A239   500507630F362905 500507630F762905  /devices/pci@0,600000/pci@0/pci@8/pci@0,1/SUNW,emlxs@1/fp@0,0/IBMtape@w500507630f762905,0
36    /dev/rmt/6st    ULT3580-TD4      00078C087D       80896/0   A239   500507630F362906 500507630F762906  /devices/pci@0,600000/pci@0/pci@8/pci@0,1/SUNW,emlxs@1/fp@0,0/IBMtape@w500507630f762906,0
38    /dev/rmt/8st    ULT3580-TD4      00078C087E       80384/0   A239   500507630F362907 500507630F762907  /devices/pci@0,600000/pci@0/pci@8/pci@0,1/SUNW,emlxs@1/fp@0,0/IBMtape@w500507630f762907,0
39    /dev/rmt/9st    ULT3580-TD4      00078C0884       79872/0   A239   500507630F362908 500507630F762908  /devices/pci@0,600000/pci@0/pci@8/pci@0,1/SUNW,emlxs@1/fp@0,0/IBMtape@w500507630f762908,0
40    /dev/rmt/10st   ULT3580-TD4      00078C0883       79616/0   A239   500507630F362909 500507630F762909  /devices/pci@0,600000/pci@0/pci@8/pci@0,1/SUNW,emlxs@1/fp@0,0/IBMtape@w500507630f762909,0
41    /dev/rmt/11st   ULT3580-TD4      00078C0881       79104/0   A239   500507630F36290A 500507630F76290A  /devices/pci@0,600000/pci@0/pci@8/pci@0,1/SUNW,emlxs@1/fp@0,0/IBMtape@w500507630f76290a,0
42    /dev/rmt/7smc   03584L32         0000078249470401 80896/1   A420         N/A             N/A        /devices/pci@0,600000/pci@0/pci@8/pci@0,1/SUNW,emlxs@1/fp@0,0/IBMtape@w500507630f762906,1

The /dev/rmt/1xx series is on one HBA, and the corresponding /dev/rmt/9xx series is on the other. NO persistent binding though.

In my searching, there is NOTHING on the web that tells how to persistently bind with the Leadville driver (emlxs).

IBM Tape Device User Guide states.....
Note: Device persistent binding is not provided on Sun HBAs, so this
persistent name approach cannot be used with the same physical drive
that is attached to multiple Sun HBA ports.

This tells me IBMtape dual path support is a non-starter with emlxs driver.

If that is the case, and I remove all tape-related entries in devlink.tab and not bind to specific /dev/rmt/xxx numbers, does the emlxs driver keep the mappings on a reboot? If not,, will TSM SANDiscovery cover me? I am thinking it does.

 

[GregE]

This is unreal to me. I've been in discussion with Sun (Oracle) on the emlxs driver being able to bind to two /dev/rmt/xxx paths so that IBMtape driver dual path failover will be useful. They are telling me that there is no way to do this with the EMLXS driver.

LPFC driver, that I currently use (and works great), is "legacy" and no longer being updated. EMLXS driver the current driver in Emulex's codestream. So, according to this info, legacy driver gives me what I need. Current driver removes the features I need. IBM XIV requires EMLXS, and we are about to move TSM to it, so this is why we are switching away from the LPFC driver.

With LPFC, I can also map primary drive devices to the HBA of my choice, so I have 10 drives and have 5 mapped to each HBA. I have a question open with Sun now asking if I will be stuck with all 10 drives on a single HBA, or if there is a mechanism to allow me to configure a more balanced load.

I will post the answer for anyone who may be considering an Emulex driver change in their environment.

 

[Mikey D]

Somehow with SUN this doesn't surprise me, probably the best way around this since they aren't providing a fix is to just have five drives zoned to one HBA port and the other five to another instead of having both see all ten.

 

[GregE]

Yep, if I can't use IBMtape dual pathing anyway, I may as well hard wire my 5 and 5. Failover was nice while it lasted, now I get to step backwards several years. Since in our new environment we just recently made the purchase, maybe IBM will refund my dual pathing support feature code purchase.