On 09/05/12 16:33, Matthew Baker wrote:
> On 05/05/12 19:54, Matthew Baker wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> On 17/09/11 20:51, James L. Evans wrote:
>>>> Not sure why that would be true. Do they use the same versions
>>>> of BackupPC?
>>>
>>> lenny was 3.1.0-4. squeeze is 3.1.0-9
>>>
>>>> It's also very likely that the nightly parameter was changed:
>>>> Maybe it was, say, 10, 1, 2, 3... and the new one> is 1, 2,
>>>> 3...; that means that the nightly is moved to a time when it is
>>>> very likely a backup is running (1AM)> > instead of a time when
>>>> it's unlikely to be doing so (10AM).
>>>
>>> Old schedule was [3.1] on lenny and squeeze. New schedule is now
>>> [16, 3.1] (with dump blacked out at 16).
>>>
>>> Given that BackupPC_nightly slows the whole machine down (not just
>>> the BackupPC_dump), I'm pretty sure this is an OS problem, not a
>>> BackupPC problem. Since changing the schedule seems to hide the
>>> problem, I'm going to ignore it for now.
>>
>> I'm having a similar issue that backuppc rsync runs are very slow
>> after the upgrade to squeeze.
>>
>> I've tried a few things:
>>
>> * swapped the network cable and port on the switch (from a wall socket).
>> * removing block sizes on rsync transfer
>> * tried sysctl tweaks in this post:
>> http://ubuntuforums.org/showthread.php?t=891695
>> * new kernel 3.2 from squeeze-backports (to see if it's a network card
>> driver)*
>> * transferring to a new empty partition and new filesystem (btrfs)
>>
>> I can manually rsync any server (e.g. my workstation in another
>> building) and get upwards of 5mbits (network negotiated at 100Mb) but
>> if I "du -s" a directory being transferred to (e.g a local server on
>> same switch) by backuppc it doesn't increase by more than
>> a few 100k each second. There might be compression/hardlink satisfying
>> the transfer but I'd still expect the directory to grow faster than that.
>>
>> Not so sure will work but is tempting:
>> * upgrade to wheezy
>> The version of backuppc is newer in wheezy but might make it unstable
>> so would rather work out what's up with squeeze.
>>
>>
>> I'm stumped. I "feels" like it's something to do with the way rsync is
>> running. Has anyone had any experience to suggest what might have
>> changed in the upgrade?
>>
>> Any advice very much welcome. TIA,
>>
>> Matt
>>
>>
>> Relevant setup pasted below:
>>
>> cmxmb@greedo:~$ lspci
>> 00:00.0 Host bridge: Intel Corporation 5100 Chipset Memory Controller
>> Hub (rev 90)
>> 00:02.0 PCI bridge: Intel Corporation 5100 Chipset PCI Express x8 Port
>> 2-3 (rev 90)
>> 00:03.0 PCI bridge: Intel Corporation 5100 Chipset PCI Express x4 Port
>> 3 (rev 90)
>> 00:04.0 PCI bridge: Intel Corporation 5100 Chipset PCI Express x4 Port
>> 4 (rev 90)
>> 00:05.0 PCI bridge: Intel Corporation 5100 Chipset PCI Express x4 Port
>> 5 (rev 90)
>> 00:06.0 PCI bridge: Intel Corporation 5100 Chipset PCI Express x8 Port
>> 6-7 (rev 90)
>> 00:07.0 PCI bridge: Intel Corporation 5100 Chipset PCI Express x4 Port
>> 7 (rev 90)
>> 00:10.0 Host bridge: Intel Corporation 5100 Chipset FSB Registers (rev 90)
>> 00:10.1 Host bridge: Intel Corporation 5100 Chipset FSB Registers (rev 90)
>> 00:10.2 Host bridge: Intel Corporation 5100 Chipset FSB Registers (rev 90)
>> 00:11.0 Host bridge: Intel Corporation 5100 Chipset Reserved Registers
>> (rev 90)
>> 00:13.0 Host bridge: Intel Corporation 5100 Chipset Reserved Registers
>> (rev 90)
>> 00:15.0 Host bridge: Intel Corporation 5100 Chipset DDR Channel 0
>> Registers (rev 90)
>> 00:16.0 Host bridge: Intel Corporation 5100 Chipset DDR Channel 1
>> Registers (rev 90)
>> 00:1c.0 PCI bridge: Intel Corporation 82801I (ICH9 Family) PCI Express
>> Port 1 (rev 02)
>> 00:1c.4 PCI bridge: Intel Corporation 82801I (ICH9 Family) PCI Express
>> Port 5 (rev 02)
>> 00:1c.5 PCI bridge: Intel Corporation 82801I (ICH9 Family) PCI Express
>> Port 6 (rev 02)
>> 00:1d.0 USB Controller: Intel Corporation 82801I (ICH9 Family) USB
>> UHCI Controller #1 (rev 02)
>> 00:1d.1 USB Controller: Intel Corporation 82801I (ICH9 Family) USB
>> UHCI Controller #2 (rev 02)
>> 00:1d.2 USB Controller: Intel Corporation 82801I (ICH9 Family) USB
>> UHCI Controller #3 (rev 02)
>> 00:1d.3 USB Controller: Intel Corporation 82801I (ICH9 Family) USB
>> UHCI Controller #6 (rev 02)
>> 00:1d.7 USB Controller: Intel Corporation 82801I (ICH9 Family) USB2
>> EHCI Controller #1 (rev 02)
>> 00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 92)
>> 00:1f.0 ISA bridge: Intel Corporation 82801IR (ICH9R) LPC Interface
>> Controller (rev 02)
>> 00:1f.2 IDE interface: Intel Corporation 82801IR/IO/IH (ICH9R/DO/DH) 4
>> port SATA IDE Controller (rev 02)
>> 00:1f.5 IDE interface: Intel Corporation 82801I (ICH9 Family) 2 port
>> SATA IDE Controller (rev 02)
>> 01:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5722
>> Gigabit Ethernet PCI Express
>> 02:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5722
>> Gigabit Ethernet PCI Express
>> 03:00.0 RAID bus controller: LSI Logic / Symbios Logic MegaRAID SAS
>> 1078 (rev 04)
>> 09:00.0 PCI bridge: Intel Corporation 6702PXH PCI Express-to-PCI
>> Bridge A (rev 09)
>> 0b:07.0 VGA compatible controller: ATI Technologies Inc ES1000 (rev 02)
>>
>> $ sudo dmidecode -t system
>> # dmidecode 2.9
>> SMBIOS 2.5 present.
>>
>> Handle 0x0100, DMI type 1, 27 bytes
>> System Information
>> Manufacturer: Dell Inc.
>> Product Name: PowerEdge T300
>> Version: Not Specified
>> Serial Number: JGBDD4J
>> UUID: 44454C4C-4700-1042-8044-CAC04F44344A
>> Wake-up Type: Power Switch
>> SKU Number: Not Specified
>> Family: Not Specified
>>
>> Handle 0x0C00, DMI type 12, 5 bytes
>> System Configuration Options
>> Option 1: NVRAM_CLR: Clear user settable NVRAM areas and set defaults
>> Option 2: PWRD_EN: Close to enable password
>>
>> Handle 0x2000, DMI type 32, 11 bytes
>> System Boot Information
>> Status: No errors detected
>>
>> $ free
>> total used free shared buffers cached
>> Mem: 4055756 3948492 107264 0 309368 2588520
>> - -/+ buffers/cache: 1050604 3005152
>> Swap: 1949692 820 1948872
>> $ uname -a
>> Linux greedo.ilrt.bris.ac.uk 3.2.0-0.bpo.2-amd64 #1 SMP Mon Apr 23
>> 08:38:01 UTC 2012 x86_64 GNU/Linux
>> $ df -h /backups
>> Filesystem Size Used Avail Use% Mounted on
>> /dev/mapper/volg0-backups
>> 1.8T 1.2T 585G 68% /backups
>> $ mount |grep backups
>> /dev/mapper/volg0-backups on /backups type reiserfs
>> (rw,noexec,nosuid,nodev,noatime,notail)
>> $ sudo ethtool eth0
>> Settings for eth0:
>> Supported ports: [ TP ]
>> Supported link modes: 10baseT/Half 10baseT/Full
>> 100baseT/Half 100baseT/Full
>> 1000baseT/Half 1000baseT/Full
>> Supports auto-negotiation: Yes
>> Advertised link modes: 10baseT/Half 10baseT/Full
>> 100baseT/Half 100baseT/Full
>> 1000baseT/Half 1000baseT/Full
>> Advertised pause frame use: Symmetric
>> Advertised auto-negotiation: Yes
>> Speed: 100Mb/s
>> Duplex: Full
>> Port: Twisted Pair
>> PHYAD: 1
>> Transceiver: internal
>> Auto-negotiation: on
>> MDI-X: Unknown
>> Supports Wake-on: g
>> Wake-on: d
>> Current message level: 0x000000ff (255)
>> Link detected: yes
>> $ sudo ethtool -S eth0
>> NIC statistics:
>> rx_octets: 28740421868
>> rx_fragments: 0
>> rx_ucast_packets: 19000689
>> rx_mcast_packets: 0
>> rx_bcast_packets: 14435
>> rx_fcs_errors: 0
>> rx_align_errors: 0
>> rx_xon_pause_rcvd: 0
>> rx_xoff_pause_rcvd: 0
>> rx_mac_ctrl_rcvd: 0
>> rx_xoff_entered: 0
>> rx_frame_too_long_errors: 0
>> rx_jabbers: 0
>> rx_undersize_packets: 0
>> rx_in_length_errors: 0
>> rx_out_length_errors: 0
>> rx_64_or_less_octet_packets: 0
>> rx_65_to_127_octet_packets: 0
>> rx_128_to_255_octet_packets: 0
>> rx_256_to_511_octet_packets: 0
>> rx_512_to_1023_octet_packets: 0
>> rx_1024_to_1522_octet_packets: 0
>> rx_1523_to_2047_octet_packets: 0
>> rx_2048_to_4095_octet_packets: 0
>> rx_4096_to_8191_octet_packets: 0
>> rx_8192_to_9022_octet_packets: 0
>> tx_octets: 627884470
>> tx_collisions: 0
>> tx_xon_sent: 0
>> tx_xoff_sent: 0
>> tx_flow_control: 0
>> tx_mac_errors: 0
>> tx_single_collisions: 0
>> tx_mult_collisions: 0
>> tx_deferred: 0
>> tx_excessive_collisions: 0
>> tx_late_collisions: 0
>> tx_collide_2times: 0
>> tx_collide_3times: 0
>> tx_collide_4times: 0
>> tx_collide_5times: 0
>> tx_collide_6times: 0
>> tx_collide_7times: 0
>> tx_collide_8times: 0
>> tx_collide_9times: 0
>> tx_collide_10times: 0
>> tx_collide_11times: 0
>> tx_collide_12times: 0
>> tx_collide_13times: 0
>> tx_collide_14times: 0
>> tx_collide_15times: 0
>> tx_ucast_packets: 9416665
>> tx_mcast_packets: 8
>> tx_bcast_packets: 29
>> tx_carrier_sense_errors: 0
>> tx_discards: 0
>> tx_errors: 0
>> dma_writeq_full: 0
>> dma_write_prioq_full: 0
>> rxbds_empty: 0
>> rx_discards: 0
>> rx_errors: 0
>> rx_threshold_hit: 0
>> dma_readq_full: 0
>> dma_read_prioq_full: 0
>> tx_comp_queue_full: 0
>> ring_set_send_prod_index: 0
>> ring_status_update: 0
>> nic_irqs: 0
>> nic_avoided_irqs: 0
>> nic_tx_threshold_hit: 0
>> mbuf_lwm_thresh_hit: 0
>> $ cat /proc/cpuinfo
>> processor : 0
>> vendor_id : GenuineIntel
>> cpu family : 6
>> model : 23
>> model name : Intel(R) Xeon(R) CPU X3323 @ 2.50GHz
>> stepping : 10
>> microcode : 0xa07
>> cpu MHz : 2500.530
>> cache size : 3072 KB
>> physical id : 0
>> siblings : 4
>> core id : 0
>> cpu cores : 4
>> apicid : 0
>> initial apicid : 0
>> fpu : yes
>> fpu_exception : yes
>> cpuid level : 13
>> wp : yes
>> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
>> mca cmov
>> pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx
>> lm constant_tsc arch_perfmon pebs bts rep_good nopl aperfmperf pni
>> dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1
>> xsave lahf_lm dts tpr_shadow vnmi flexpriority
>> bogomips : 5001.06
>> clflush size : 64
>> cache_alignment : 64
>> address sizes : 36 bits physical, 48 bits virtual
>> power management:
>>
>> processor : 1
>> vendor_id : GenuineIntel
>> cpu family : 6
>> model : 23
>> model name : Intel(R) Xeon(R) CPU X3323 @ 2.50GHz
>> stepping : 10
>> microcode : 0xa07
>> cpu MHz : 2500.530
>> cache size : 3072 KB
>> physical id : 0
>> siblings : 4
>> core id : 1
>> cpu cores : 4
>> apicid : 1
>> initial apicid : 1
>> fpu : yes
>> fpu_exception : yes
>> cpuid level : 13
>> wp : yes
>> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
>> mca cmov
>> pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx
>> lm constant_tsc arch_perfmon pebs bts rep_good nopl aperfmperf pni
>> dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1
>> xsave lahf_lm dts tpr_shadow vnmi flexpriority
>> bogomips : 4999.96
>> clflush size : 64
>> cache_alignment : 64
>> address sizes : 36 bits physical, 48 bits virtual
>> power management:
>>
>> processor : 2
>> vendor_id : GenuineIntel
>> cpu family : 6
>> model : 23
>> model name : Intel(R) Xeon(R) CPU X3323 @ 2.50GHz
>> stepping : 10
>> microcode : 0xa07
>> cpu MHz : 2500.530
>> cache size : 3072 KB
>> physical id : 0
>> siblings : 4
>> core id : 2
>> cpu cores : 4
>> apicid : 2
>> initial apicid : 2
>> fpu : yes
>> fpu_exception : yes
>> cpuid level : 13
>> wp : yes
>> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
>> mca cmov
>> pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx
>> lm constant_tsc arch_perfmon pebs bts rep_good nopl aperfmperf pni
>> dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1
>> xsave lahf_lm dts tpr_shadow vnmi flexpriority
>> bogomips : 4999.98
>> clflush size : 64
>> cache_alignment : 64
>> address sizes : 36 bits physical, 48 bits virtual
>> power management:
>>
>> processor : 3
>> vendor_id : GenuineIntel
>> cpu family : 6
>> model : 23
>> model name : Intel(R) Xeon(R) CPU X3323 @ 2.50GHz
>> stepping : 10
>> microcode : 0xa07
>> cpu MHz : 2500.530
>> cache size : 3072 KB
>> physical id : 0
>> siblings : 4
>> core id : 3
>> cpu cores : 4
>> apicid : 3
>> initial apicid : 3
>> fpu : yes
>> fpu_exception : yes
>> cpuid level : 13
>> wp : yes
>> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
>> mca cmov
>> pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx
>> lm constant_tsc arch_perfmon pebs bts rep_good nopl aperfmperf pni
>> dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1
>> xsave lahf_lm dts tpr_shadow vnmi flexpriority
>> bogomips : 4999.99
>> clflush size : 64
>> cache_alignment : 64
>> address sizes : 36 bits physical, 48 bits virtual
>> power management:
>>
>>
>> - --
>
> Just following up with this as this is preventing any backups from
> completing. The first dump job process which was spawned has been
> running at 100% CPU since the 5th of May and the BackupPC_dump process
> is showing messages like this when straced:
>
> $ sudo strace -p 3488
> Process 3488 attached - interrupt to quit
> lseek(7, 142595739, SEEK_SET) = 142595739
> read(7,
> "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"...,
> 14261) = 14261
> lseek(7, 142595739, SEEK_SET) = 142595739
> read(7,
> "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"...,
> 14261) = 14261
> lseek(7, 142595739, SEEK_SET) = 142595739
> read(7,
> "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"...,
> 14261) = 14261
> lseek(7, 142595739, SEEK_SET) = 142595739
>
> ad infinitum...
>
> any pointers or suggestions would be very welcome.
By looking at the size of directories growing under the clients new
directory I could see that /proc was most active. This particular client
didn't have a specific BackupFilesExclude config so I've set a global
default to:
$Conf{BackupFilesExclude} = {
'*' => [
'/tmp/*',
'/proc/*',
'/sys/*'
]
};
Restarted and retried this client and it finished an incremental run! =]
Over half of the other clients have /proc already excluded. Will see
over night if this helps those that don't.
Cheers,
Matt
--
Matthew Baker :: Senior Systems Administrator :: University of Bristol
+----------------------------------------------------------------------+
| Infrastructure, Systems and Operations it-sysops AT bristol.ac DOT uk |
| T: Berkeley Square: +44(0)117 3314325 (Mon, Thur & Fri) |
| T: Computer Centre: +44(0)117 3317467 (Tue, Wed) |
| A: Uni of Bristol, Computer Centre, Tyndal Ave, Bristol. BS81UD |
+----------------------------------------------------------------------+
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
|
