Move node to another server is very slow

[Sunhillow]

Hello together
This morning I started "export node" to another server. The node contains ca. 6 million files and 255 TB of data.
Source is TSM 7.1.3 on AIX, destination is TSM 7.1.3 on Linux. Both servers are at the same location, network link is 2*10 GB on each. The goal is to shut down and switch off the AIX server after completion and move all tapes to a vault.

During the first 2 hours nothing happened, then data transfer begun ... slowly. Start time was 09:30 CEST, now it is 15:45 CEST and only 90 GB of data has been transferred so far!

Would it be faster to do node replication and delete the source node as soon as replication is ready? At least multiple streams can be defined for node replication. I did noth think this procedure is so slow ...

 

[marclant]

I think node replication only does multiple streams if there are multiple filespaces to replicated, but not 100% sure of that. Keep in mind that in addition to transfer 255 TB, it has 6 million objects to read on the source database, and 6 million to insert on the target database. So, here are all the things that can impact your performance.
- speed of the DB disks on source and target
- speed of the storage pool disks/tapes on source and target
- network

The key is to figure out which one is the bottleneck. You can use iostat to look at the you want the "read avgserv" and the "write avgserv" to be under 10 msec for spinning disks, under 3 msec for SSD.

Maybe increase the TCPWindowsize on the target server, that will require a restart and also restart the export.

 

[Sunhillow]

Thank you marklant!
Target DB resides on SSD, source on SAS. The target server already reports 600 mio files, including copy pools I think. So 6 mio in addition would not be too much.

The source has two filespaces with most of data inside plus six filespaces with ca. 1 GB.

I will wait until tomorrow and then decide how to continue ...

 

[marclant]

So 6 mio in addition would not be too much.

I just meant that there is more than just the data copy to do with the export, there's all the transactions needed to insert those objects in the database.

With the databases, you also benefit from multiple database directories with 1 LUN for each directory. At least 4 directories for DB2 less than 2TB. 8 dir for less than 4 TB, 12 dir for anything larger than 4 TB. SAS is not as good as SSD, but as long as you don't have any dedup pool, it should work alright.

 

[Sunhillow]

Today I tried Export again, but with a fast diskpool as first pool on the target server. Now the rate is > 240 GB/h what is well in the expected range.

The source system uses 4 LUNs with 6.5 GB each on SAN (SAS),
the target uses 8 LUNs with 100 GB each on SAN (SSD)
and because we store on tape, there is no dedup activated.