Re: Performance bottleneck tips

Hi,

I'll do you one example of a node with a network bottleneck, from our act log
I took:

04/10/02   14:39:22      ANE4952I (Session: 4419, Node: SI_REMCO_PIPC)  Total

                          number of objects inspected:  130,542

04/10/02   14:39:22      ANE4954I (Session: 4419, Node: SI_REMCO_PIPC)  Total

                          number of objects backed up:      891

04/10/02   14:39:22      ANE4958I (Session: 4419, Node: SI_REMCO_PIPC)  Total

                          number of objects updated:          0

04/10/02   14:39:22      ANE4960I (Session: 4419, Node: SI_REMCO_PIPC)  Total

                          number of objects rebound:          0

04/10/02   14:39:22      ANE4957I (Session: 4419, Node: SI_REMCO_PIPC)  Total

                          number of objects deleted:          0

04/10/02   14:39:22      ANE4970I (Session: 4419, Node: SI_REMCO_PIPC)  Total

                          number of objects expired:        796

04/10/02   14:39:22      ANE4959I (Session: 4419, Node: SI_REMCO_PIPC)  Total

                          number of objects failed:           0

04/10/02   14:39:22      ANE4961I (Session: 4419, Node: SI_REMCO_PIPC)  Total

                          number of bytes transferred:    21.55 MB

04/10/02   14:39:22      ANE4963I (Session: 4419, Node: SI_REMCO_PIPC)  Data

                          transfer time:                1,399.48 sec

04/10/02   14:39:22      ANE4966I (Session: 4419, Node: SI_REMCO_PIPC)
Network
                          data transfer rate:           15.76 KB/sec

04/10/02   14:39:22      ANE4967I (Session: 4419, Node: SI_REMCO_PIPC)
Aggregate
                          data transfer rate:         11.58 KB/sec

04/10/02   14:39:22      ANE4968I (Session: 4419, Node: SI_REMCO_PIPC)
Objects
                          compressed by:                    0%

04/10/02   14:39:22      ANE4964I (Session: 4419, Node: SI_REMCO_PIPC)
Elapsed
                          processing time:            00:31:45

As you can see the data transfer time is: 1,399 sec or 23:20 minutes. The
remainder of the total processing time (31:45 minutes) is spend by the node
examining objects (and this one only has 130k files). Since the time it takes
to dl. a fs last state from the server to be able to compare it with the
current state of the fs, is small compared to the time it takes to walk the
fs, this can be neglected as a factor (even with very little bandwidth, like
in this example).

If your data transfer time is very small compared to the total time, the
bottleneck is in the client node, examining objects...

AS you can see, the network transfer rate here is about 15.76 KB/s, which is
absolutely unacceptable on a lan, in this case it is something we have to live
with... ;)


--
Met vriendelijke groeten,
Met vriendelijke groeten,

Remco Post

SARA - Stichting Academisch Rekencentrum Amsterdam    http://www.sara.nl
High Performance Computing  Tel. +31 20 592 8008    Fax. +31 20 668 3167

"I really didn't foresee the Internet. But then, neither did the computer
industry. Not that that tells us very much of course - the computer industry
didn't even foresee that the century was going to end." -- Douglas Adams