BackupPC-users

Re: [BackupPC-users] Max file sizes

2010-01-25 11:38:38
Subject: Re: [BackupPC-users] Max file sizes
From: "Michael Stowe" <mstowe AT chicago.us.mensa DOT org>
To: "General list for user discussion, questions and support" <backuppc-users AT lists.sourceforge DOT net>
Date: Mon, 25 Jan 2010 10:37:27 -0600
There are two points at which you'll run into problems:
1) At the 2GB barrier
2) At the point at which you exceed smbclient's timeout value, which
naturally depends on the speed of your network, hard drives, and the size
of your timeout value

> thank you for that!
>
> any experiences with SMB and large files?  with updated smbclients, when
> are
> we likely to run into problems?
>
> thanks a lot!
>
> On Sun, Jan 24, 2010 at 11:17 AM, Pedro M. S. Oliveira <
> pmsoliveira AT gmail DOT com> wrote:
>
>>
>> Currently I have some vmware machine files with over 120GB and the
>> backups
>> go smooth.
>> I've used tar and rsync but I've had trouble when I had compression > 6.
>> The specs for the backup server are Dell rack server 2x dual core, 6x
>> 750GB
>> SATAII with raid 5, 6TB ram, raid controller has 512 MB.
>>
>> This server backups up about 17 servers and 2 desktops
>>    *  31 full backups of total size 2071.27GB (prior to pooling and
>> compression),
>>    * 195 incr backups of total size 1841.14GB (prior to pooling and
>> compression).
>> # Pool is 1505.93GB comprising 5443879 files and 4369 directories (as of
>> 1/24 05:11),
>> # Pool hashing gives 252 repeated files with longest chain 18,
>> # Nightly cleanup removed 55176 files of size 220.26GB (around 1/24
>> 05:11),
>> # Pool file system was recently at 69% (1/24 19:01), today's max is 84%
>> (1/24 01:00) and yesterday's max was 84%.
>>
>> The server has a backup concurrency of 4 (during nights)
>> The load average is about 10-12 during night
>>
>>
>> The same backuppc setup with lower specs would give trouble from time to
>> time and had to limit the concurrency to 2 and compression to 2
>> Cheers,
>> Pedro
>>
>>
>> On Friday 22 January 2010 12:33:36 Simon Fishley wrote:
>> > Hi All
>> >
>> > The FAQ here
>> http://backuppc.sourceforge.net/faq/limitations.html#maximum_backup_file_sizes
>> > suggests there are limitations to the maximum size of a file that can
>> > be backed up, specifically when using GNUtar or smbclient.
>> >
>> > I have several users running VirtualBox under Ubuntu and, regularly,
>> > these virtual machine files exceed 10GB for an individual file. We
>> areid
>> > using rsync to backup the data and the FAQ suggests more testing is
>> > needed to be certain of the max file size rsync can manage.
>> >
>> > Has anyone done any of this testing? Can anyone share with me how they
>> > are backing up virtual machine files?
>> >
>> > Thanks
>> > Simon
>> >
>> >
>>


------------------------------------------------------------------------------
Throughout its 18-year history, RSA Conference consistently attracts the
world's best and brightest in the field, creating opportunities for Conference
attendees to learn about information security's most important issues through
interactions with peers, luminaries and emerging and established companies.
http://p.sf.net/sfu/rsaconf-dev2dev
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/