Re: [BackupPC-users] Backing up many small files

> -----Original Message-----
> From: Adam Goryachev [mailto:mailinglists AT websitemanagers.com DOT au]
> Sent: Tuesday, February 05, 2013 10:28 AM
> To: backuppc-users AT lists.sourceforge DOT net
> Subject: Re: [BackupPC-users] Backing up many small files

Hi, and thanks for the swift reply!


> Have you done at least two FULL backups since you enabled the
> checksum-seed option? If not, stop now, and wait until you have.

I have eighteen full backups online for this particular machine.


> Check the following during an incremental backup:
> 1) Memory/swap used on both backup server and the backup client. If you
> are using all available memory, or see memory being paged in/out (use
> vmstat) then you need to upgrade RAM on that machine, or find a way to
> backup a smaller number of files (split the client into multiple shares
> or multiple machines, etc).

Thanks. Are there any particular limits/numbers I should be aware of, i.e. the 
rule-of-thumb kind?


> 2) Check disk performance on the backup client. You have a single SATA
> drive on the client, and this will be slow, you are doing a lot of
> seeks, not just one big read. Can you enable noatime on the client
> (probably)? This would decrease the amount of seek and writes on the
> single SATA drive.

Noatime on client; Just checked - not set. Setting. Thanks for the reminder!

As for the single drive, I can't do much about that. It's an instrument 
computer and not really allowed to change the config or the service 
support-people won't be too happy about it.


> 3) Check disk performance on the backup server

Any best practices here?


> 4) Check CPU on the backup server, if you have compression enabled, this
> will really slow things down, consider to disable compression (though
> this well mess with the pool).

Is this the (GUI) Edit Config/Backup settings/CompressLevel=3 you're referring 
to?


> 5) Check bandwidth between the two (least likely to be the culprit, but
> worth checking).

user@BPC-server ~/ [0]# lftp -e 'pget 
http://ftp.sunet.se/pub/os/Linux/distributions/centos/6/isos/x86_64/CentOS-6.3-x86_64-bin-DVD1.iso'
`/pub/os/Linux/distributions/centos/6/isos/x86_64/CentOS-6.3-x86_64-bin-DVD1.iso',
 
got 193027041 of 4289386496 (4%) 6.99M/s eta:11m

Speeds varies around 7 M(Mbyte? Mbit?)/s. I guess it's good enough for a 
100Mbps-connection.


> BTW, are you sure the backup server has 2TB with 4 drives in RAID0 ?
> That suggests that any one of those 4 drives fail, and you lose ALL of
> your backups and pool etc... You might confirm you are using RAID0 and
> not linear, and also check the stripe size. If you are backing up lots
> of small files, then you want the stripe size to be about the same size
> as your file size. If your files are between 1 and 2kB each, then you
> would want a stripe size of 4k, not the current linux default of 512k.
>
> I would suggest RAID10 if you want any sort of resilience....

I was a bit wrong here I see; three drives and 1,4 TB. All seem active. It 
would seem I also added a drive on the PATA-port, in addition to the 
SATA-ports. I think the reason for using a PATA-drive at the time was the mobo 
only had two SATA-ports and I needed more space, thus adding a slower 
PATA-drive as well. The actual BPC-server was rather old even at the beginning 
when it was converted to a backup-server.

Maybe PATA would slow things down a bit as well?

user@BPC-server ~/ [0]# cat /proc/mdstat
Personalities : [raid0]
md0 : active raid0 sdb1[2] sda1[1] hdb1[0]
      1465151616 blocks 128k chunks

unused devices: <none>

I'm assuming the above mentioned 128 kB chunks are the same as stripe sizes 
and can't be changed to the 4 kB size you mention w/o a reformat. Correct?

Anyway, I know. It's a calculated risk using raid0.
I'm figuring as these are just casual backups (users always copy their 
personal data using Winscp to their homefolders, which is being backed up on 
another more resilient BPC), there's no real need for redundancy - just plenty 
of space.


-- 
/Sorin

smime.p7s
Description: S/MIME cryptographic signature

------------------------------------------------------------------------------
Free Next-Gen Firewall Hardware Offer
Buy your Sophos next-gen firewall before the end March 2013 
and get the hardware for free! Learn more.
http://p.sf.net/sfu/sophos-d2d-feb

_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/