1.) To the 43 TB File pool and once the 43 TB filepool fills up it
migrates to Tape.
2.) 20 GB
3.) No collocation anywhere in the system at this time. Eventually I
will be setting up group collocation for DR use so I can restore Higher
priority data first then second tier and so on. Typically restores are
a file here or there. We rarely have to restore full servers but yes
all the data would come mostly from the 43 TB file pools.
4.) Once Daily
I did kind of try something similar to what you were mentioning with
collocation based on changes.
The 43 TB is actually comprised of approximately 2 - 13 TB Disk Trays.
2 - 6 TB Luns of Sata disk on a san and a 5 TB disk tray.
I tried tiering each drive as a separate storage pool each flowing into
the next one using the fastest disk as the first and slowest as the
last. Then I set migration delays so only old data would move down to
the next storage pool. The problem is if I get to many processes
hitting each drive I really slowed down the drive. Also like current
situation I am in we have to much new data coming in and to much old
data going out that I can't get my migration and reclamation processes
to run sequentially in my time window.
I couldn't find enough information to optimize the system to run quickly
and cleanly so abandoned that path.
Also I wanted to keep everything together so that once deduplication
becomes usable in Tivoli I can implement it right away.
I have also tried running multiple simultaneous processes such as
migrate with a duration set and then later run a wait=yes process to
finish up but occasionally they overlap which just causes more failures.
For example I migrate stg diskpool lo=0 dur=180 and then later in the
script I run migrate stg diskpool lo=0 wait=yes to finish off the
cleanup.
Jeremy
-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:ADSM-L AT VM.MARIST DOT EDU] On Behalf Of
Sam Rawlins
Sent: Wednesday, December 03, 2008 3:55 PM
To: ADSM-L AT VM.MARIST DOT EDU
Subject: Re: [ADSM-L] Maintenance Processes - Scheduling optimization
I have some questions that might clear up your situation:
1) You say the 1.1 TB Disk and 3 TB File pools migrate. Where to? The 43
TB File pool, or tape?
2) What is the volume size of these file pools?
3) What is collocation set to in the 3 TB File pool and the long-term 43
TB File pool?
4) How often do you perform a DB backup? Perhaps more to the point, how
often would you like to perform a DB backup?
As a pre-reply to 3), its a good practice to set collocation=no in the 3
TB File pool to speed up backups. Then, set
collocation=group|node|filespace in the 43 TB File pool; this will speed
up restores. I base these on the idea that backups go from client to 3
TB File pool, then to 43 TB File pool, where they are most likely to be
restored from. Is this correct?
If you can dedicate some real consideration to solving this problem, you
might look at how you can set up the File Volume sizes and collocation
compared with filespace sizes for your nodes who's data changes most
often.
I'm sure a clever scheme of defining collocation groups based on
percent_data_changed per backup can help.
On Wed, Dec 3, 2008 at 2:40 PM, goc <gooooc AT gmail DOT com> wrote:
> oh, yes i would also like to hear some ideas about that, especially
> since we are running cca. 80 database archive logs backups every 3
> hours,
> and some other kung-fu ... and i've seen expire inventory and db
> backups running at the
> same time almost on daily basis.
>
> ok, i have 3 db backups daily, one on disk, other on tape device
> class and third on remote tsm
>
> hi, my name is goran and i have flaw in my backup system.
>
> On Wed, Dec 3, 2008 at 10:25 PM, Conradt, Jeremy
> <JConradt AT ashleyfurniture DOT com> wrote:
> > I have recently forced myself to admit we have a major flaw in our
> > backup system. What was happening and may be happening to many
> > other people is we had maintenance processes, reclaims, migrations
> > and so on happening on fileclass volumes after or even during the
database backup.
> > The problem with this is if the database were to crash or corrupt
> > after its backup any maintenance work that happened after the
> > beginning of the backup will cause issues with the recovery.
> >
> > For example X file exists on Filevolume1 at the beginning of the
> > database backup but after the backup migration or reclamation moves
> > it to Filevolume2. The database corrupts we restore the database
> > which doesn't know anything about Filevolume2 and now it can't find
> > Filevolume1 which forces us to restore Filevolume1 from offsite.
> > Basically just a lot of work to get everything back in sync.
> >
> > I have been working on trying to get all of our backup stg, migrate
> > stg and reclaim stg to run in a single script sequentially but there
> > is insufficient time in the day to complete successfully. We end up
> > overlapping into the next backup window which just slows everything
> > down.
> > The Copy pool "Offsite" tapes are fine because I have the stg set
> > with a delay period of 1 day but I don't want to tie up file volume
> > space for a day if I don't have to.
> > I am wondering if anyone has worked their way through this issue and
> > developed a good set of scripts to get everything running cleanly
> > and in proper order.
> >
> > System Stats
> > TSM Server version 5.3.5
> > Windows 2003 SP2
> >
> > 1.1 TB Disk pool 15K Fiber channel san disk with DISK class volumes
> > Files smaller than 3GB are backed up to this location.
> > This pool is migrated to 0% every day.
> >
> > 3 TB file class volumes on 10K Fiber Channel san disk
> > Files larger than 3GB are backed up to this location.
> > This pool is migrated to 50% every day.
> >
> > 43 TB file class volumes on multiple disk drives primarily SATA.
> > Long term storage or data.
> >
> > In general we have about 32 TB of regular backup data and about 60
> > TB of Archive data on tape.
> > We back up approximately 2 TB of data daily.
> > At this time all of our backup data remains on disk.
> >
> > If anyone has any questions about how we have anything setup and
> > working please let me know.
> > If anyone has any suggestions on how to resolve any problems they
> > see in our system please let me know.
> > Thanks,
> > Jeremy
> >
> >
> >
> >
>
>
>
> --
>
> Jim Bishop - "Golf is played by twenty million mature American men
> whose wives think they are out having f...
>
--
Sam Rawlins
|
