Re: [Networker] Incrementals and Indexes ?

ah, ok, it's not as bad as I thought then. It is an Enterprise product after
all, so backing up filesystems is the intent, but saying that, it's a pretty
sloppy bug, a slap on the wrist for EMC there, and I can see that it has
serious implications for your situation.

thanks for the documentaion pointer. I'll get reading ... right after I've
finished moving all of my critical documents to this wonderful "/dev/nul"
that you mention (an infinite storage folder you say, the wonders of Unix
never cease ...)


On Thu, Apr 3, 2008 at 1:28 AM, Mathew Harvest <
Mathew.HARVEST AT sis.qld.gov DOT au> wrote:

> Doing full backups to /dev/nul will solve your need for an infinitely
> sized disk...
>
> The nappies here got more than wet when management found out about this
> bug
>
>
>
> You shouldn't see this problem
> if you are using the change journal AND you are doing full file system
> backups (ie D:\ rather than D:\user\data\here\) apparently specifying a
> sub directory changes the way the network interacts with the change
> journal.
>
> You will probably see this behaviour if
> 1. You are backing up the file system and you don't have change journal
> enabled or
> 2. You are backing up the file system and you are using VSS (apparently
> using VSS disables the change journal or
> 3. You are backing up a sub directly of the file system
>
> Our problem is that if we turn off VSS then we loose our ability to deal
> with open files, and if we don't then we have a recovery nightmare if we
> have to perform a full file system recovery
>
> The admin guide has a pretty good explanation of how the change journal
> works (in my version 7.3.2 its is in appendix D section 10)
>
>
>
> -----Original Message-----
> From: EMC NetWorker discussion [mailto:NETWORKER AT LISTSERV.TEMPLE DOT EDU] 
> On
> Behalf Of Roy Wiseman
>  Sent: Thursday, 3 April 2008 8:45 AM
> To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
> Subject: Re: [Networker] Incrementals and Indexes ?
>
> thanks for confirming this Michael. wow ... that's a fairly large and
> significant bug. The scenario that I described of a user wanting a
> folder
> structure back probably does happen quite regularly, and would lead to a
> *
> management-nappy-wetting-tantrum* level of escalation (quite serious)
> that I
> wouldn't want to be around to explain. I'll do full backups of
> *everything*every 2 hours from now on with my infinitely-sized,
> infinite-bandwidth disk
> library until it's fixed then. thanks.
>
> Can you explain the overall process on how incrementals are processed
> when
> NetWorker is actually working properly ? does it first take an index of
> the
> files to be backed up, and then compare that with the last backup index
> ?
> how does it do the compare, filenames only ? crc ? some other method ??
> (obviously the last step it currently takes is to flush the index down
> the
> toilet. But hopefully EMC will get out SP2 to fix that asap ...).
>
>
>
>
> On Thu, Apr 3, 2008 at 12:17 AM, Mathew Harvest <
> Mathew.HARVEST AT sis.qld.gov DOT au> wrote:
>
> > Its not you ... its Networker ... engineering apparently have a ticket
> -
> > LGTpa35113 (I think) for this problem...
> >
> > In the mean time you just need to ask your users to not rename any
> > directories or move any files, or you could do daily full backups...
> >
> > Mat.
> >
> >
> > -----Original Message-----
> > From: EMC NetWorker discussion [mailto:NETWORKER AT LISTSERV.TEMPLE DOT EDU]
> On
> > Behalf Of Roy Wiseman
> > Sent: Thursday, 3 April 2008 8:01 AM
> > To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
> >  Subject: Re: [Networker] Incrementals and Indexes ?
> >
> > Thanks Michael,
> >
> > I've refreshed the GUI and no change unfortunately. I've been seeing
> > this
> > behaviour for a few weeks and it's only now that I thought I'd ask
> > someone about it. Version is NetWorker Management Console version
> > 3.4.1.Build.335 based on NetWorker version 7.4 SP1.Build.335.
> >
> > I do a full backup. Then I rearrange the folder structure. Then I
> change
> > say
> > 2 files. The 2 changed files will be caught in the incremental but
> > nothing
> > else, and when I open up NetWorker User and point the Browse Time at
> the
> > day
> > of the incremental, I only see the 2 changed files and nothing else
> (no
> > directory structure or files). Am I doing something really naive here
> ?
> > These are fairly pristine installations, no options changed and the
> > group
> > and pool are just by the defaults. If I flip the Browse Time back a
> day
> > I
> > see everything, but for today, for the incremental, I see only the two
> > changed files. Any ideas what I'm doing wrong here ?
> >
> > Thanks,
> >
> > Roy.
> >
> > On Wed, Apr 2, 2008 at 11:03 PM, Michael Nored <mrnored AT netzero DOT net>
> > wrote:
> >
> > > Roy,
> > > Works for me I see the file in the recovery GUI under the new folder
> > name.
> > > Did you refresh your GUI in between backups?
> > >
> > > Michael Nored
> > > Consultant
> > > Nored Consulting, LLC
> > >
> > > M: (407) 399-8162
> > > T:  (352) 350-7142
> > > E: mnored AT Netzero DOT net
> > >
> > >
> > > -----Original Message-----
> > > From: EMC NetWorker discussion
> [mailto:NETWORKER AT LISTSERV.TEMPLE DOT EDU]
> > On
> > > Behalf Of Roy Subs
> > > Sent: Wednesday, April 02, 2008 4:03 PM
> > > To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
> > > Subject: [Networker] Incrementals and Indexes ?
> > >
> > > Hi,
> > >
> > > I've got a question about how NetWorker processes incrementals.
> > > I backed up a new folder, D:\Stuff, so an inital full save set was
> > made of
> > > 20,000 files taking up 5 GB.
> > > I then created a new subfolder, D:\Stuff\Temp, and I moved ALL files
> > from
> > > D:\Stuff into D:\Stuff\Temp. So, I now have D:\Stuff\Temp which
> > contains
> > > all the files, and D:\Stuff which has no files, but just the
> subfolder
> > > Temp
> > > in it. I then started an incremental backup.
> > > Final result: *nothing* was backed up.
> > >
> > > So what is the process for the incr ? Is it :
> > > 1. NetWorker indexes all the files in the location to be
> incrementally
> > > backed up ?
> > > 2. then, it creates CRC's and cross-references that to the index of
> > the
> > > original full ?
> > > 3. then, only files that do not have a match in the existing full
> > index
> > > from step 2 ?
> > >
> > > BUT, and this is sorta important, if I go to NetWorker User, and
> look
> > at
> > > available files from that incr backup, NOTHING is there. This makes
> > sense
> > > of course, as there were no file changes incrementally backed up,
> but
> > what
> > > if you massively altered the directory structure and you lost that
> ??
> > You
> > > would not be able to reconstruct the changed structure !
> > >
> > > e.g. a user called Joe Tucci spends 8 hours on Wednesday
> reorganising
> > his
> > > work folder. he does not EDIT a single file, but that 8 hours of
> > > organisation work IS work (ok, maybe Joe should've had a better
> > > organisation system for his files before now, but still ...). Then,
> on
> > > Thursday morning an incremental backup happens at 3:33 am. On
> Thursday
> > > morning, a NAS box explodes wiping out the entire data center (as
> that
> > was
> > > the particular revision of Celerra that they made out of
> > nitro-glycerine.
> > > an easy design mistake to make. oops). But, Joe thinks "that's ok !,
> > the
> > > backups were made, so all my folder reorganisation is safe !" ...
> but
> > > actually, Joe's out of luck, as NOTHING was saved of his
> > reorganisation as
> > > NetWorker decided that no files had changed so it simply backed up
> > NOTHING
> > > (so all the 8 hours of folder reorganisation that Joe did is lost).
> > Joe
> > > can
> > > get all his files back from the Wednesady morning backup, but he's
> > going
> > > to
> > > have to do all that 8 hours of reorganisation again by hand.
> > >
> > > Is that right ?? I'm just quite curious, as it might be useful to
> > backup
> > > changed folder structures (i.e. fair enough don't backup the files
> as
> > > nothings changed, but DO at least backup an index of the changed
> > folder
> > > structure every night). Is there some option to do this ? And also,
> > what
> > > is
> > > the exact process by which NetWorker determines what will and will
> not
> > be
> > > backed up ?
> > >
> > > Thanks,
> > >
> > > Roy.
> > >
> > > To sign off this list, send email to listserv AT listserv.temple DOT edu
> and
> > type
> > > "signoff networker" in the body of the email. Please write to
> > > networker-request AT listserv.temple DOT edu if you have any problems with
> > this
> > > list. You can access the archives at
> > > http://listserv.temple.edu/archives/networker.html or
> > > via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
> > >
> > >
> > >
> > >
> >
> > To sign off this list, send email to listserv AT listserv.temple DOT edu and
> > type "signoff networker" in the body of the email. Please write to
> > networker-request AT listserv.temple DOT edu if you have any problems with
> this
> > list. You can access the archives at
> > http://listserv.temple.edu/archives/networker.html or
> > via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
> >
> >
> >
> >
> > ********************************* DISCLAIMER
> > *********************************
> > The information contained in the above e-mail message or messages
> (which
> > includes any attachments) is confidential and may be legally
> privileged.  It
> > is intended only for the use of the person or entity to which it is
> > addressed.  If you are not the addressee any form of disclosure,
> copying,
> > modification, distribution or any action taken or omitted in reliance
> on the
> > information is unauthorised.  Opinions contained in the message(s) do
> not
> > necessarily reflect the opinions of the Queensland Government and its
> > authorities.  If you received this communication in error, please
> notify the
> > sender immediately and delete it from your computer system network.
> >
> >
> >
> >
>
> To sign off this list, send email to listserv AT listserv.temple DOT edu and
> type "signoff networker" in the body of the email. Please write to
> networker-request AT listserv.temple DOT edu if you have any problems with 
> this
> list. You can access the archives at
> http://listserv.temple.edu/archives/networker.html or
> via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
>
>
>
>
> ********************************* DISCLAIMER
> *********************************
> The information contained in the above e-mail message or messages (which
> includes any attachments) is confidential and may be legally privileged.  It
> is intended only for the use of the person or entity to which it is
> addressed.  If you are not the addressee any form of disclosure, copying,
> modification, distribution or any action taken or omitted in reliance on the
> information is unauthorised.  Opinions contained in the message(s) do not
> necessarily reflect the opinions of the Queensland Government and its
> authorities.  If you received this communication in error, please notify the
> sender immediately and delete it from your computer system network.
>
>
>
>

To sign off this list, send email to listserv AT listserv.temple DOT edu and 
type "signoff networker" in the body of the email. Please write to 
networker-request AT listserv.temple DOT edu if you have any problems with this 
list. You can access the archives at 
http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER