Re: [ADSM-L] TDP for VE; CBT backup size versus OS modified file size

Hi Stefan,

Thanks for the tip. We indeed didn't exclude the swap file. We will improve our 
configuration.

But for the comparison it was included in listing the modified files as in the 
backup made.

If the entire file was changed, we expect that the modified files and the 
backup were the same size, but the backup is much bigger.

Regards,
Ruud

-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:ADSM-L AT VM.MARIST DOT EDU] On Behalf Of 
Stefan Folkerts
Sent: Monday, June 01, 2015 8:36 AM
To: ADSM-L AT VM.MARIST DOT EDU
Subject: Re: [ADSM-L] TDP for VE; CBT backup size versus OS modified file size

Ruud,

>After all, the fact that a file is modified does not mean that the
>entire
file has changed (meaning that all CBT blocks need to be backed up).

Regarding this statement, when you open, edit and save an office file (for
example) it rewrites the entire thing to disk, even if you edit or add a single 
line (so I was told).
This is the case with most edits I believe except for databases and maybe some 
other exceptions that don't open all corresponding data and rewrite the whole 
thing to disk when you save.

Regards,
   Stefan



On Sat, May 30, 2015 at 9:35 AM, Stefan Folkerts <stefan.folkerts AT gmail DOT 
com>
wrote:

> Hi Ruud,
>
> Did you put the Windows swap file on a seperate disk and exclude it
> from the VE backup?
> If not this is part of the VE backup and might be (a part) of the size
> increase you are seeing if Windows swap space is in use.
> A file backup using the backup archive client might also exclude other
> data (from the local include exclude list or a server client option
> set) that you might need to take into account.
>
> Regards,
>    Stefan
>
>
>
> On Fri, May 29, 2015 at 4:51 PM, Meuleman, Ruud <
> ruud.meuleman AT tatasteel DOT com> wrote:
>
>> Hi all,
>>
>> We administer a vSphere 5.5 environment using TDP for VE as our
>> backup solution. TDP for VE uses CBT to make incremental-forever
>> backups of our VMs, meaning that it only backs up the entire VM once;
>> all subsequent backups are incrementals. We backup our VMs once a
>> day. We are currently investigating why certain Windows VMs in our
>> vSphere environment generate huge incremental backups. During our
>> investigation we hit on something we don't understand. As a
>> quick-and-dirty test, we did a file scan on a VM, to list all files that had 
>> been modified on the VM during the last 24 hours.
>> We then added up the total file size of all those modified files.
>> Then, we compared this combined file size with the size of the
>> incremental TSM-VE backup for that day. we repeated this test on a
>> number of VMs, smaller as well as larger ones. We had expected the
>> combined file size to be much larger than the size of the incremental
>> backup. After all, the fact that a file is modified does not mean
>> that the entire file has changed (meaning that all CBT blocks need to
>> be backed up). We expected a CBT backup to be more efficient,
>> size-wise, than an incremental file-based backup. Instead, we found
>> out that on all VMs, the incremental TSM-VE backup was consistently
>> 1.5 to twice the size of the combined modified file size, exactly the 
>> opposite of the result we expected.
>>
>> We've tried to think of a few things that could cause this discrepancy.
>> 1) In-guest disk defrag. This would change the blocks without
>> changing the files, messing up the way CBT works. However, there are
>> no scheduled or unscheduled defrags on our VMs.
>> 2) The files on the VM are smaller than the CBT blocks. That could
>> cause a small file to mark a larger CBT block as changed. However, as
>> we understand it, CBT blocks are usually quite small (and not the
>> same as VMFS
>> blocks)
>>
>> What are we missing here? Is there some other process that changes
>> the VMDK storage blocks of my Windows VMs without changing the actual
>> files? Is my quick-and-dirty file scan too simplistic? I really hope
>> that someone can explain this to me, thanks!
>>
>> Kind Regards,
>> Ruud Meuleman
>>
>> *********************************************************************
>> *
>>
>> This transmission is confidential and must not be used or disclosed
>> by anyone other than the intended recipient. Neither Tata Steel
>> Europe Limited nor any of its subsidiaries can accept any
>> responsibility for any use or misuse of the transmission by anyone.
>>
>> For address and company registration details of certain entities
>> within the Tata Steel Europe group of companies, please visit
>> http://www.tatasteeleurope.com/entities
>>
>> *********************************************************************
>> *
>>
>
>
**********************************************************************

This transmission is confidential and must not be used or disclosed by anyone 
other than the intended recipient. Neither Tata Steel Europe Limited nor any of 
its subsidiaries can accept any responsibility for any use or misuse of the 
transmission by anyone.

For address and company registration details of certain entities within the 
Tata Steel Europe group of companies, please visit 
http://www.tatasteeleurope.com/entities

**********************************************************************