Spectrum Protect Plus - Hyper-V backup performance

cfhoffmann

ADSM.ORG Senior Member
Joined
Jun 2, 2005
Messages
116
Reaction score
3
Points
0
Location
Sydney, Australia
PREDATAR Control23

With version 10.1.2 I'm seeing less than 10 MB/s backup throughput of VMs on Hyper-V. Is anyone else backing up Hyper-V and seeing similar performance ?
 
PREDATAR Control23

Have the same performance issue but with DP for VE, 8.1.6. Here's a full backup of a VM:

Total number of bytes inspected: 400.00 GB
Total number of bytes transferred: 373.13 GB
Data transfer time: 49,619.75 sec
Network data transfer rate: 7,884.98 KB/sec
Aggregate data transfer rate: 7,812.10 KB/sec
Objects compressed by: 0%
Total data reduction ratio: 6.72%
Elapsed processing time: 13:54:42

Client side dedup & compression turned off.
 
PREDATAR Control23

The network data transfer rate is low. This points to something upstream from the client. Anything between the network card on the client and the final storage pool where the data is stored, so lots of things to look at. A bit too complex to troubleshoot in an online forum.
 
PREDATAR Control23

In my case network data transfer is not the issue. Tested separately and no issues on that side.
 
PREDATAR Control23

In my case network data transfer is not the issue. Tested separately and no issues on that side.
It is an issue, it's slow. Note that I didn't say you had a network issue, but that the network data transfer rate is slow, and that's indicative of an issue upstream from the client up in the direction of the server, including the server. All potential culprits:
  • NIC and network settings on client
  • network infrastructure
  • NIC and network settings on server
  • server DB disk, if DB updates are slow, that slows the backup. Should be SSD with latency < 3 msec. Should have 1 DB directory/LUN for every 500 GB of DB size.
  • server active log disk, that also slows down DB updates, which slows backup. Should be SSD with latency < 3 msec
  • storage pool disks, same thing if data is written slow, it slows down the backup. Latency should be < 10 msec for spinning disk
  • CPU and memory on the server can be issues too
 
PREDATAR Control23

Marclant, I'm not sure if I'm reading this instrumentation right, but to me it looks like most of the time is spent on VM I/O.

VM I/O

Time that is spent reading and writing data to and from the VIX Disk Library for Virtual Disk Development Kit (VDDK) disks for a VM. Performance can vary depending on whether thin or thick-provisioned disks are used and if the disks are lazy zeroed.

So does this mean that source disks are slow since the most time is spent there?

From the dsminstr.log:

Code:
Section          Actual(sec)    Average(msec)    Frequency used    Bytes Involved
-----------------------------------------------------------------------------------
Sleep                  3.047       1015.7            3            0
TCP Read              341.642       3673.6           93            0
VM Snapshot Complete  92.313      92313.0            1            0
VM Send Data          783.259          0.6      1290039    334820502184
VM Query               3.440          0.6         5521            0
VM VCM lock           39.293          0.0     20428965            0
VM Transaction        311.759         57.3         5444            0
VM I/O                69899.206         54.5      1281888            0
Thread Wait            1.313        119.3           11            0
Other                 39.459          0.0            0            0

-----------------------------------------------------------------------------------

Also, stats are terrible:

04/03/2019 15:01:49 Successful Full VM backup of 'Hyper-V' Virtual Machine 'OCR'
mode: 'Incremental Forever - Full'
target node name: 'HV-NODE'
data mover node name: 'HV-NODE-M'
04/03/2019 15:01:49 Statistics for Virtual Machine 'OCR'.
04/03/2019 15:01:49 Total number of objects inspected: 1
04/03/2019 15:01:49 Total number of objects backed up: 1
04/03/2019 15:01:49 Total number of objects updated: 0
04/03/2019 15:01:49 Total number of objects rebound: 0
04/03/2019 15:01:49 Total number of objects deleted: 0
04/03/2019 15:01:49 Total number of objects expired: 0
04/03/2019 15:01:49 Total number of objects failed: 0
04/03/2019 15:01:49 Total number of objects skipped: 0
04/03/2019 15:01:49 Total number of objects encrypted: 0
04/03/2019 15:01:49 Total number of objects grew: 0
04/03/2019 15:01:49 Total number of retries: 0
04/03/2019 15:01:49 Total number of bytes inspected: 500.00 GB
04/03/2019 15:01:49 Total number of bytes transferred: 264.23 GB
04/03/2019 15:01:49 Data transfer time: 61,140.76 sec
04/03/2019 15:01:49 Network data transfer rate: 4,531.66 KB/sec
04/03/2019 15:01:49 Aggregate data transfer rate: 4,480.03 KB/sec
04/03/2019 15:01:49 Objects compressed by: 0%
04/03/2019 15:01:49 Total data reduction ratio: 47.16%
04/03/2019 15:01:49 Elapsed processing time: 17:10:45
 
PREDATAR Control23

So does this mean that source disks are slow since the most time is spent there?
That's what this seems to indicate. To word it differently, it's the category that the most time is spent in, so to make the backup go faster, the amount of time spent there has to be reduced.
 
Top