Hi,

Christian Völker wrote on 2015-09-28 22:02:26 +0200 [Re: [BackupPC-users] Slow 
transfer via rsync?]:
> >> Why is it still so slow?
> [...]
> traceroute to 192.168.1.3 (192.168.1.3), 30 hops max, 60 byte packets
>  1  10.12.0.1 (10.12.0.1)  344.955 ms  355.805 ms  355.818 ms
>  2  infra2 (192.168.1.3)  359.722 ms  359.737 ms  364.983 ms

are you serious? Where do you get over 1/3 second delay from? I'd expect about
a tenth of that (just tested over a UDP OpenVPN link). This is where I'd look.
It obviously *is* latency related. What do you get from a traceroute from one
VPN endpoint to the other (not through the VPN)?

Oh, is this traceroute in parallel to the running rsync exchange? Not that it
should make much difference, considering the low bandwidth usage ...

> [...]
> >  Is the firewall blocking (any) ICMP traffic between the two endpoints? 
> No. As you can see in the above commands it appears everything if fine.

No, I don't really see that. I see that ICMP echo request/reply are working,
not that PMTU discovery is working (but maybe I missed it).

> > Can you use tcpdump to look at the characteristics
> > of the connection (e.g. is there a constant (slow) exchange, or are there
> > hangs, timeouts, retransmissions)?
> See above- there is an exchange but I am unsure how to read the output
> of tcpdump in detail.

It wasn't about the detail, it was about the timing. The timestamps seem to
indicate a rhythm corresponding to the RTT with something like 1340 bytes of
[raw IP] data sent every 300ms - faster at the end (because more is sent
before the ACK arrives). The fragment you quoted seemed to show data transfer
only from "infra2" to "bu" (and ACKs without data in the other direction), so
- at this point - it doesn't seem to be the rsync protocol exchange limiting
the speed. It also doesn't *appear* to be the TCP window size [yet], because
the last three packets from infra2 to bu are sent without waiting for the ACK
whereas the first ones apparently aren't.
With an RTT of around about 333 ms, you would need a window size of 213KB for
a theoretical throughput of 5 Mbit/s (assuming data is only transfered in one
direction without waiting for rsync protocol responses from the other side).

Please avoid line wrapping in future tcpdump quotes.

> [...]
> UDP for OpenVPN:
> [...]
> tls-client

The server side could have more options that affect speed ...

> > [...] the *first* backup of a host is usually special in that all data
> > not already in the pool (which might in this case mean all data) needs to
> > be compressed. 
> Prior to be compressed it should have been transferred. Does BackuPC
> compress "on-the-fly" or after all data is transferred?  However- CPU
> cycles are close to zero (see above).

On the fly. The bandwidth of the data stream is not high enough to make
compression use significant CPU resources, but it *will* take *some* time,
which *might* add another few (tens of) ms to a RTT if the File::RsyncP
implementation compresses data before sending a protocol reply the remote
end is waiting for. I don't know the protocol exchange well enough to know
whether this will have a measurable effect (if any).

> > [...] I'd give it some more time before starting to worry.
> I wouldn't worry if I at least one of the parameters (IO, CPU, memory,
> network) would show some usage. But none of them really does.

It's about latency, not about saturation.

Regards,
Holger

------------------------------------------------------------------------------
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/