On 08/06/2012 04:12 PM, Arbogast, Warren K wrote:
There is a Linux fileserver here that serves web content. It has 21 million
files in one filesystem named /ip. There are over 4,500 directories at the
second level of the filesystem. The server is running the 6.3.0.0 client, and
has 2 virtual cpus and 16 GB of RAM. Resourceutilization is set to 10, and
currently there are six client sessions running. I am looking for ways to
accelerate the backup of this server since currently it never ends.
The filesystem is NFS mounted so a journal based backup won't
work. Recently, we added four proxy agents, and are splitting up the
one big filesystem among them using include/exclude statements. Here
is one of the agent's include/exclude files.
exclude /ip/[g-z]*/.../*
include /ip/[a-f]*/.../*
... You say "proxy agents", but it's not clear what you mean by this.
__Since we added the proxies the proxy backups are copying many
thousands of files, as if this were the first backup of the server
as a whole. Is that expected behavior?
I see two possible arrangements you might have implemented.
Possibility one: where you had BIGFS-NODE which was taking a long
time, you now have BIGFS-NODEAF with the config above, and
BIGFS-NODEGL which is defined to include G through L, etc.
In this case, each BIGFS subnode will need to re-back-up its initial
incremental, and thereafter you should see normal change rates.
I will note that this is unlikely to accelerate your wall-clock time;
if you've got resourceutilization 10, you've probably got 5+ threads
walking the FS, you've probably moved your bottleneck to IOPS on your
NAS as it tries to pull the metadata to satisfy the FS walk. 20
threads won't do that faster.
Possibility two: you have four agents with different include/exclude
statements but using the same BIGFS node.
In this case you are running a charlie-foxtrot formation. One process
is backing stuff up while another is expiring the very same stuff. If
you're doing this, you should stop it immediately, go back to one
agent, and complete an incremental, because you have your backups in
an indeterminate state.
__Recently, the TSM server database is growing faster than it
usually does, and I'm wondering whether there could be any
correlation between the ultra long running backup, many thousands of
files copied, and the faster pace of the database growth.
This symptom is what makes me think you're doing the latter. If one
process is adding stuff while another throws it away, the result is a
rapidly growing tail of inactive versions, capped by (I think)
VERDELETED. I don't know off the top of my head if an excluded file
is retained at VEREXISTS or VERDELETED. Interesting.
- Allen S. Rout
| 