Ok I found the summary info:
START_TIME: 2002-01-02 03:00:14.000000
END_TIME: 2002-01-02 03:00:54.000000
ACTIVITY: BACKUP
ENTITY: UNXR
EXAMINED: 234
BYTES: 16321897
START_TIME: 2002-01-02 03:00:21.000000
END_TIME: 2002-01-02 04:20:10.000000
ACTIVITY: BACKUP
ENTITY: UNXR
EXAMINED: 366
BYTES: 9471366179
(1.5 MB/s)
START_TIME: 2002-01-02 03:01:05.000000
END_TIME: 2002-01-02 04:24:47.000000
ACTIVITY: BACKUP
ENTITY: UNXR
EXAMINED: 155
BYTES: 27776810539
(3.1 MB/s)
START_TIME: 2002-01-02 03:03:17.000000
END_TIME: 2002-01-02 04:20:17.000000
ACTIVITY: BACKUP
ENTITY: UNXR
EXAMINED: 23
BYTES: 15313256977
(3.16 MB/s)
Your comments about the multiple backups (or streams) is what I suspect. I'm
kinda surprised by the low throughput I see here.
Miles
----------------------------------------------------------------------------------------------
-------------------
Miles Purdy
Miles Purdy
System Manager
Farm Income Programs Directorate
Winnipeg, MB, CA
purdym AT fipd.gc DOT ca
ph: (204) 984-1602 fax: (204) 983-7557
"If you hold a UNIX shell up to your ear, can you hear the C?"
-------------------------------------------------------------------------------------------------
----------------------
>>> Robin_Sharpe AT BERLEX DOT COM 31-Dec-01 10:11:56 PM >>>
>>> Robin_Sharpe AT BERLEX DOT COM 31-Dec-01 10:11:56 PM >>>
Well, how to lookup individual sessions: The short answer is SELECT * FROM
SUMMARY WHERE ENTITY='nodename' AND ACTIVITY='BACKUP'
Multi streamed backups are a great enhancement, but there are some
subtleties.
With a single streamed backup, you have one session and one statistics
report on standard output (if running from a script). If running from a
unixcommand line it comes out on your terminal screen. If running from a
GUI it is in a window, but has a slightly different format (if I remember
correctly).
With a multi streamed backup, you have several sessions, as you have
stated. But you still get a single statistics report. In the GUI you can
see the individual sessions, but on the command line version, there is no
indication that it's a multi stream. And this where I think the problem
lies.
So, when you do the select from summary, you should get several records
that start at the same time (or thereabouts). These would be the sessions
that make up the multi streamed backup. I think you have to calculate the
elapsed time (end-start)... For the.... ***FLASH*** light bulb just went
on over my head!
This is why it isn't accurate: Each session on a multi stream backup
will be a different elapsed time (probably). Since they are running
concurrently, the elapsed time for the backup is of course the end time of
the latest session minus the start time of the earliest. But each session
has a data transfer time that may mor may not overlap the other sessions.
The total data transfer is the sum of those of each session... so it could
conceivably be more than the elapsed time. Still doesn't seem to make
sense though.... (light bulb is dimming now.... and less than an hour of
2001 remains!)
I'm really getting curious about this now... guess I'll have to do some
tests when I get back to work on Wednesday.
Hope everyone has a Safe & Happy New Year!
Robin Sharpe
Berlex Labs
Miles Purdy
<PURDYM@FIPD.
GC.CA> To: ADSM-L AT VM.MARIST DOT EDU
cc: (bcc: Robin Sharpe/WA/USR/SHG)
12/30/01 Subject:
10:25 AM Re: Please Explain (again)
Please
respond to
"ADSM: Dist
Stor Manager"
Hi guys,
I do not think any times are wrong nor is there a bug, see example.
PS: How do I look up individual sessions for a given backup?
Example:
Here's how I thought the numbers were arrived at:
say processing starts at time zero and runs for 1 hour, 3600 s, just to
make it easy, and we backup 100 GB. We also use two streams 50 GB total
each, and they run concurrently. %75 of the time is spent sending data, %25
processing overhead.
So:
total time: 3600 s
total bytes: 100 GB
aggregate is: 100 * 1024 mb / 3600 = 28 MB /s
data transfer time: .75 * 3600 * 2 = 5400 s
network throughput is: 100 * 1024 / 5400 = 18 MB / s
I think what I meant to say was the network pipe is really _wide_. The
network can sustain multiple streams running at full speed. The limiting
speed factor may be the server or it may be the network, but I don't think
it matters where the speed bottleneck is. So I still don't think it is a
bug.
Miles
----------------------------------------------------------------------------------------------
-------------------
Miles Purdy
Miles Purdy
System Manager
Farm Income Programs Directorate
Winnipeg, MB, CA
purdym AT fipd.gc DOT ca
ph: (204) 984-1602 fax: (204) 983-7557
"If you hold a UNIX shell up to your ear, can you hear the C?"
-------------------------------------------------------------------------------------------------
----------------------
>>> Robin_Sharpe AT BERLEX DOT COM 28-Dec-01 9:56:57 PM >>>
>>> Robin_Sharpe AT BERLEX DOT COM 28-Dec-01 9:56:57 PM >>>
As I said a while ago, I think it's a bug. I'm guessing that this is a
multi stream backup (I think that has already been established), and the
data transfer time is the total of all of the sessions, but the elapsed
time is for only one session... can you confirm that Miles? Hopefully you
still have records for those sessions in the summary table.
Robin Sharpe
Berlex Labs
"Zlatko
Krastev/ACIT"
<acit@ATTGLOB To: ADSM-L AT VM.MARIST DOT EDU
AL.NET> cc: (bcc: Robin Sharpe/WA/USR/SHG)
Subject:
12/27/01 Re: Please Explain (again)
07:30 PM
Please
respond to
"ADSM: Dist
Stor Manager"
Sorry, I am taking my words back. Have a look again at the times reported
Data transfer time: 8 261.61 sec
Elapsed processing time: 01:25:00
This 8261 seconds is definitely much more than 1h25m (5100s) and one of the
times is erroneous. Divided by wrong value you're getting one of the rates
wrong.
Zlatko Krastev
IT Consultant
Zlatko Krastev/ACIT AT attglobal DOT net>
22.12.2001 23:55
Sent by: Zlatko Krastev/ACIT<acit AT attglobal DOT net>
To: "ADSM: Dist Stor Manager" <ADSM-L AT VM.MARIST DOT EDU>
cc:
Subject: Re: Please Explain
If your network capacity "greatly exceeds the clients ability to send data"
then your network rate should "greatly exceed" the data read (on the node)
and write (on the server) rate. Consequently aggregate rate has also to be
"greatly exceeded". The only explanation I can find to the numbers you're
seeing is that the speed of the SP Switch is incorrectly counted in MHz not
in MB/s. And later those MHz are converted as usual serial Ethernet in
MB/s. Because SP Switch running at low frequency gives high throughput your
network rate is displayed wrong.
Zlatko Krastev
IT Consultant
Miles Purdy <PURDYM AT FIPD.GC DOT CA> on 20.12.2001 17:26:13
Please respond to "ADSM: Dist Stor Manager" <ADSM-L AT VM.MARIST DOT EDU>
To: ADSM-L AT VM.MARIST DOT EDU
cc:
Subject: Re: Please Explain
I don't think it is a bug. I think is because my network (SP Switch)
capacity greatly exceeds the clients ability to send data (even though is
an S80). If in effect I'm running 2,3,4,5 backups concurrently.
12/19/01 14:25:28 ANE4952I (Session: 14442, Node: UNXP) Total
number of objects inspected: 135 487
12/19/01 14:25:28 ANE4954I (Session: 14442, Node: UNXP) Total
number of objects backed up: 2 309
12/19/01 14:25:28 ANE4958I (Session: 14442, Node: UNXP) Total
number of objects updated: 0
12/19/01 14:25:28 ANE4960I (Session: 14442, Node: UNXP) Total
number of objects rebound: 0
12/19/01 14:25:28 ANE4957I (Session: 14442, Node: UNXP) Total
number of objects deleted: 0
12/19/01 14:25:28 ANE4970I (Session: 14442, Node: UNXP) Total
number of objects expired: 13 138
12/19/01 14:25:28 ANE4959I (Session: 14442, Node: UNXP) Total
number of objects failed: 0
12/19/01 14:25:28 ANE4961I (Session: 14442, Node: UNXP) Total
number of bytes transferred: 60.53 GB
12/19/01 14:25:28 ANE4963I (Session: 14442, Node: UNXP) Data
transfer time: 8 261.61 sec
12/19/01 14:25:28 ANE4966I (Session: 14442, Node: UNXP) Network
data transfer rate: 7 682.73 KB/sec
12/19/01 14:25:28 ANE4967I (Session: 14442, Node: UNXP) Aggregate
data transfer rate: 12 445.33 KB/sec
12/19/01 14:25:28 ANE4968I (Session: 14442, Node: UNXP) Objects
compressed by: 0%
12/19/01 14:25:28 ANE4964I (Session: 14442, Node: UNXP) Elapsed
processing time: 01:25:00
Miles
----------------------------------------------------------------------------------------------
-------------------
Miles Purdy
Miles Purdy
System Manager
Farm Income Programs Directorate
Winnipeg, MB, CA
purdym AT fipd.gc DOT ca
ph: (204) 984-1602 fax: (204) 983-7557
"If you hold a UNIX shell up to your ear, can you hear the C?"
-------------------------------------------------------------------------------------------------
----------------------
>>> Robin Sharpe AT BERLEX DOT COM 20-Dec-01 8:20:26 AM >>>
>>> Robin Sharpe AT BERLEX DOT COM 20-Dec-01 8:20:26 AM >>>
I don't see how aggregate rate could exceed network rate. Aggregate is
Bytes Transferred divided by Elapsed Time, and Network is Bytes Transferred
divided by Data Transfer Time.... what were those values in the session
where Aggregate was greater than Network? How could Data Transfer Time be
greater than Elapsed Time? Must be a bug!
Robin Sharpe
Berlex Labs
Miles Purdy
<PURDYM@FIPD.
GC.CA> To: ADSM-L AT VM.MARIST DOT EDU
cc: (bcc: Robin Sharpe/WA/USR/SHG)
12/20/01 Subject:
08:44 AM Re: Please Explain
Please
respond to
"ADSM: Dist
Stor Manager"
The network transfer rate is the time is takes to send the file to be
backed up to the TSM server.
The aggregate is the total KB backed up / the total time.
The difference is the processing time. The time to contact the TSM server
and check if the file needs to be backed up.
Interestingly enough, my aggregate usually exceeds the network, I was
asking yesterday if any one else sees this.
Miles
----------------------------------------------------------------------------------------------
-------------------
Miles Purdy
Miles Purdy
System Manager
Farm Income Programs Directorate
Winnipeg, MB, CA
purdym AT fipd.gc DOT ca
ph: (204) 984-1602 fax: (204) 983-7557
"If you hold a UNIX shell up to your ear, can you hear the C?"
-------------------------------------------------------------------------------------------------
----------------------
>>> rouzen AT UNIV.HAIFA.AC DOT IL 20-Dec-01 4:45:16 AM >>>
>>> rouzen AT UNIV.HAIFA.AC DOT IL 20-Dec-01 4:45:16 AM >>>
Hi
I run a full backup of a Netware 5 with Tivoli client 4.2.0 here the
statistics:
12/18/2001 13:58:12 Total number of objects inspected: 33,310
12/18/2001 13:58:12 Total number of objects backed up: 33,104
12/18/2001 13:58:12 Total number of objects updated: 0
12/18/2001 13:58:12 Total number of objects rebound: 0
12/18/2001 13:58:12 Total number of objects deleted: 0
12/18/2001 13:58:12 Total number of objects expired: 0
12/18/2001 13:58:12 Total number of objects failed: 3
12/18/2001 13:58:12 Total number of bytes transferred: 1.33 GB
12/18/2001 13:58:12 Data transfer time: 138.38 sec
12/18/2001 13:58:12 Network data transfer rate: 10,099.94 KB/sec
12/18/2001 13:58:12 Aggregate data transfer rate: 802.25 KB/sec
12/18/2001 13:58:12 Objects compressed by: 0%
12/18/2001 13:58:12 Elapsed processing time: 00:29:02
Can anybody explain why my Network data transfer rate is so high but my
Aggregate data transfer rate is low !!!!!!!!!!!!
T.I.A Regards
Robert Ouzen
E-mail: rouzen AT univ.haifa.ac DOT il
|
