ADSM-L

Re: Performance Large Files vs. Small Files

2001-02-21 16:09:06
Subject: Re: Performance Large Files vs. Small Files
From: bbullock <bbullock AT MICRON DOT COM>
Date: Wed, 21 Feb 2001 14:09:26 -0700
        I'll take a look into that option again to see if it will work for
me.

Thanks,
Ben Bullock
UNIX Systems Manager

> -----Original Message-----
> From: Petr Prerost [mailto:pprerost AT HTD DOT CZ]
> Sent: Wednesday, February 21, 2001 2:43 AM
> To: ADSM-L AT VM.MARIST DOT EDU
> Subject: Re: Performance Large Files vs. Small Files
> 
> 
> Hello ,
> did you check -fromdate and -fromtime ( and -totime and 
> -todate ) restore
> parameters ?
> 
> Regards
>           Petr
> 
> ----- Puvodní zpráva -----
> Od: "bbullock" <bbullock AT MICRON DOT COM>
> Komu: <ADSM-L AT VM.MARIST DOT EDU>
> Odesláno: 21. února 2001 1:16
> Predmet: Re: Performance Large Files vs. Small Files
> 
> 
> >         Point well taken Steve. Your classification of the 
> nature of the
> > data is basically correct except for a twist. On the day the data is
> > written, it is extracted by other programs that analyze the 
> data to spot
> > flaws and trends in the manufacturing process. Depending on 
> what it finds,
> > they may then need to delve deeper into the data to analyze and fix
> > production flaws. So their argument is that without that 
> data online, they
> > have no idea if the chips we manufactured a couple of hours 
> ago are good
> or
> > not.
> >
> >         True, that after a couple of days the data is infrequently
> accessed,
> > and after about a week, the data is rarely accessed, that's why they
> delete
> > it after 30 days. But restoring just the files that were 
> newly backed up
> the
> > night before is not possible... is it? I don't think a point in time
> restore
> > will do that...
> >
> >         I like the idea of renegotiating the SLA (service 
> level agreement)
> > with the customer so that their expectations are set and my butt is
> covered.
> > Thanks for the advice.
> >
> > Thanks,
> > Ben Bullock
> > UNIX Systems Manager
> >
> >
> > > -----Original Message-----
> > > From: Steve Harris [mailto:Steve_Harris AT HEALTH.QLD.GOV DOT AU]
> > > Sent: Tuesday, February 20, 2001 4:43 PM
> > > To: ADSM-L AT VM.MARIST DOT EDU
> > > Subject: Re: Performance Large Files vs. Small Files
> > >
> > >
> > > Ben
> > > >>> bbullock <bbullock AT MICRON DOT COM> 21/02/2001 8:21:34 >>>
> > > >>>Big Snip
> > >         This one nightmare host now has over 20 million 
> files (and an
> > > unknown number of directories) across 10 filesystems. We have
> > > found from
> > > experience, that any more than about 500,000 files in any
> > > filesystem means a
> > > full filesystem restore would take many hours. Just to
> > > restore the directory
> > > structure seems to take a few hours at least. I have told the
> > > admins of this
> > > host that it is very much unrecoverable in it's current
> > > state, and would
> > > take on the order of days to restore the whole box.
> > >
> > >         They are disappointed that an "enterprise backup
> > > solution" can't
> > > handle this number of files any better. They are willing to
> > > work with us to
> > > get a solution that will both cover the daily "disaster
> > > recovery" backup
> > > need for the host and the long term retentions they desire.
> > > >>> remainder snipped
> > >
> > > I would debate whether the host is "unrecoverable".  It seems
> > > that this data is write once/read seldom in nature.  In that
> > > case, in the event of a disaster the priorities are
> > > 1. get the server back to the state where it can write more files
> > > 2. get back any individual required file in reasonable time
> > > TSM can provide both of those objects.  If a full restore is
> > > required it *can* be spread over days because most of the
> > > data will never be needed.  You will of course need to
> > > negotiate wth your users exactly what is urgent and needs to
> > > be restored immediately and maybe have some canned macro to
> > > do this in the heat of an emergency.
> > >
> > > Regards
> > >
> > >
> > > Steve Harris
> > > AIX and ADSM Admin
> > > Queensland Health, Brisbane Australia
> > >
>