The preliminary information:
- 3590B drives have data rate 9 MB/s native, 27 MB/s with IBM
claimed 3:1 compression;
- 3590E/H have 14 MB/s native, 42 MB/s with IBM claimed 3:1
compression (Orville, LTO drives have 15 MB/s native, 3590x have less but
with better compression algorithm);
- the throughput of FC 6227 is 1062,5 Mb/s which is 132,8 MB/s
theoretically but in real life due to latency and protocol overhead is
about 100 MB/s;
- the throughput of RS/6000 Sxx servers' RIO loop is 500 MB/s duplex
and each I/O drawer's slots are spread across 4 PCI buses (you didn't ask
but these may also become a bottleneck);
- the driver for ESS (Subsystem Device Driver) provides failover
*and* true load balancing;
- the driver for 3590x (Atape.driver) provides only failover (in
last few 7.x versions) and *does not* provide true load balancing. The so
called "dynamic load balancing" is not for each data block or I/O
operation but for device open/close (source "IBM TotalStorage Tape Device
Drivers, Installation and User s Guide");
- according to IBM access of both SAN disk and SAN tape through same
HBAs is supported in configurations which include RS/6000 with FC 6227,
2105 ESS, 3494 library with 3590 drives and IBM 2109 (Brocade Silkworm)
SAN switches. You meet all the requirements except SAN switches (at the
moment I have not this info) but I expect you meet it as well.
Now the math:
- if you will move client node compressed data (I recall you are
compressing your R/3 data at Sun E10k), even 6-7 drives per adapter can be
fine (100 MB/s : 14 MB/s ~= 7,14);
- if data will be compressed by the drive (which I expect is problem
you are trying to solve) no more than 2 drives should be accessed through
single adapter. Do not mess burst rate (transfers to drive's buffer) with
data rate (transfers down to the media, compressed or not). Former is 100
MB/s (because 3590 drives have 1 Gb FC interface) while latter is 14 MB/s
or more (without or with compression). Several people have reported higher
compression ratios (70-80 MB/s per drive) and you might even need an
adapter for each drive (which probably is not affordable);
- all HBAs and drives should be connected through switch(es) (I
expect they are but is worth to mention it). Connection in FC-AL topology
will limit everything to single 100 MB/s and will render all questions
discussed here unnecessary.
Plain comparison will say that 5 drives to a 100 MB/s (FC) adapter will be
as good as 2 drives to a 40 MB/s (SCSI) adapter. To get "same" performance
as one-to-one SCSI attachment no more than 2-3 drives should be connected.
Load-balancing of tape drives across all four HBAs would be the best
approach. 8 drives across 4 HBAs means 2 drives/HBA which is nearly
perfect. ESS access can be also ballanced across all 4 HBAs. Being in
supported configuration simultaneous ESS reads and 3590/3494 writes ought
to be fine. You know will this server be used for TSM only or for another
application as well. In latter case you have to include that application's
access to ESS in the claculation.
I am not sure but am under impression FC 6227 are 32-bit PCI adapters (and
am sure they are 33 MHz ones). Thus simultaneous reads and writes using
one adapter will be limited to 133 MB/s. Also simultaneous storage
operations across HBAs combined with network activity might hit 500 MB/s
limit of the RIO loop. So count all adapters in the drawer very carefully.
At the end processors might be also limiting factor - someone have to
drive all those LAN&SAN I/Os. According to IBM S70 allows maximum of four
6227 HBAs but I suspect this is for 12-way system. 4-way configuration
might be unable to drive all four 6227 at full speed.
Your numbers for SCSI-attached drives are rather low. Even for already
compressed data (no speed improvement due to drive compression) they are
less than the possbile maximum - 9 MB/s = 32,4 GB/h (for 3590Bs), 14 MB/s
= 50,4 GB/h (for 3590Es). Check why migration is slow - diskpool
performance, adapter placement limitations (PCI buses, drawers, another
adapters' activity), processor constraints, many small files, etc.
Hope this helps.
Zlatko Krastev
IT Consultant
"Cook, Dwight E" <DWIGHT.E.COOK AT SAIC DOT COM>
Sent by: "ADSM: Dist Stor Manager" <ADSM-L AT VM.MARIST DOT EDU>
31.12.2002 18:35
Please respond to "ADSM: Dist Stor Manager"
To: ADSM-L AT VM.MARIST DOT EDU
cc:
Subject: 3590 (E or H) drives per FC interface (6227) ???
I've looked through the archives and can't find much.
I've waded through IBM's web sites and can't find specifics...
Has anyone found any recomendations on max number of 3590's (FC attached)
per FC card on host ?
Environments I'm looking at are 7017-S70's that have a max of 4 FC cards
(6227's) per system.
I'm already driving my ESS storage with 2, that would leave me with 2
cards
to drive 8 tape drives.
(or maybe mix everything together and let the device drivers balance the
loads ????)
So is anybody driving 4 3590's off a single FC adapter and if so how are
your data transfer rates look with 1, 2, 3 & all 4 drives running ?????
Math says
1Gb (1024Mb/sec) FC card is 128 MB/sec or 450 GB/hr
100 MB/sec burst data transfer rate of 3590 is 351 GB/hr
so really 1 card per 1 drive but 4 cards could drive 5 drives...
NOW...
With 3590-B1A's & E1A's that are SCSI attached (one drive per one scsi
adapter) I see migrations run at
B1A'a 22-ish GB/hr
E1A's 32-ish GB/hr
Now SCSI attached drives have a burst rate of 40 MB/sec 140 GB/hr
So on my E1A's I see migrations run at 23% of the burst rate
So if I expect the same from FC, migrations across FC would be at 23% of
100
MB/sec or 23% of 351 GB/hr or 80 GB/hr if I have one drive per one FC
card...
So with all that said, a single FC adapter should be able to drive five
and
a half (5.5-ish) FC 3590-E1A's or H1A's and run at least as good as SCSI
attached at a one-to-one atachment... right???
Anyone have any thoughts...
(other than I need a vacation)
Dwight E. Cook
Software Application Engineer III
Science Applications International Corporation
509 S. Boston Ave. Suite 220
Tulsa, Oklahoma 74103-4606
Office (918) 732-7109
|
