[Networker] How can data, or number of files, cause extremely slow write

If you have a very large number of files on a given file system can thisslow down the backup write speeds, not the actual completion times, butthe speed of the writing to tape itself?

We have an ext3 file system on a Linux box that is extremely painful toback up because it chugs along at less than 100 KB/sec. I finally gaveup and canceled the backup. I was running a level full. The strangething is that all the rest of the data on that box, located on the otherext3 file systems, backs up at normal speed, even to the same tape anddrive (LTO-3).

As far as I can tell, there's nothing suspect with the drive or tape,and there's no process running on there that's sucking up memory oraccessing the affected data in such a manner that it would cause thisissue. Compression is enabled on the drives.

It's consuming less than half the disk space (300 out of 750 GB).However, that particular file system is also consuming over 10 millioninodes. Nasty! As a result, running commands like 'du' or 'find' cantake a long time to complete. I'm not sure if the data is binary. Itcould be, and this would obviously adversely affect compression, so Iwould expect that we might see slower write speeds as a result. Also, ifyou're only sending a single save stream to tape, it will be slower thanmultiplexing multiple streams. All that makes sense, but I've never seenanything this slow, even when running a single save set, except inextreme cases where there was a switch problem, NIC issue, or extremelyslow client, but again, the other backups on that host do run at normalspeed.

I think NetWorker walks through save sets before it backs them up so itknows if a file's size has changed once it starts to back it up? Thatsaid, it might take it a while to create that preliminary list on a filesystem with that many inodes, but once it's done, and it starts theactual backups, would the number files, and/or the type of data, affectthe actual speed that the backups perform at???? I can see that it mighttake it longer to complete the backups, but how would this create suchslow write speeds? Is this surprising given that number of inodes?

Does anyone have any suggestions on how we might be able to speed thisup? The only thing I can think of is to identify the minimal number ofareas under there that need to be backed up and skip the rest, maybebreak those minimal backups into multiple save sets to multiplex thedata and force faster write speeds? I could turn compression off on thedrive, but I don't wanna leave it that way as it generally works to ourfavor on most data. Would be a pain to dedicate a non-compression driveto just one chunk of data.


Thanks for any suggestions.

George


--
George Sinclair
Voice: (301) 713-3284 x210

- The preceding message is personal and does not reflect any official orunofficial position of the United States Department of Commerce -

- Any opinions expressed in this message are NOT those of the US Govt. -

To sign off this list, send email to listserv AT listserv.temple DOT edu and type 
"signoff networker" in the body of the email. Please write to networker-request 
AT listserv.temple DOT edu if you have any problems with this list. You can access the 
archives at http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER

[Networker] How can data, or number of files, cause extremely slow write speeds?