Question from IBM about the type of daily reports you use or would like

danthomp

Newcomer
Joined
Sep 26, 2002
Messages
3
Reaction score
0
Points
0
Website
Visit site
PREDATAR Control23

Hello,

My name is Dan Thompson, and I am an SE for IBM who specializes in TSM/Spectrum Protect.

I am trying to gather information on what kind of reports users of the software would like to see. I am familiar with the excellent SQL references users have compiled and shared, but I am trying to ask a more fundamental question: What key reports do people need, what form and delivery options will they need and who is the intended recipient?

In discussions with other internal IBMers with a similar skill set, we suspect that a high percentage of needs can be met by a small number of reports, perhaps as few as 10.

So, if you have a few minutes, please respond to this thread with the following information:
1) What report(s) do you need?
2) What is the fundamental task you want the report to enable? (for example: I want to know how much of the planned capacity of my TSM/Protect server is being utilized, for any and all resources).
3) Does the fundamental task require information from multiple sources?
4) What is the scope of the report? Multiple servers, single server, single node, single schedule, etc.
5) What form do you want this report (pre-formatted like the daily OC report, tabular, chart, don't care)?
6) Who is the intended recipient (TSM/Protect admins, operations, mid-level mgmt, upper mgmt, etc.)?
7) What is the distribution method (email, post to a web server and email link, pdf, spreadsheet, etc.)?
8) How often is the report needed (daily, weekly, other)?
9) If you want to give an example of what you are describing, please feel free to share (SQL query, snapshot or mock-up).

If it is not obvious enough, I want to make certain I don't misunderstand your requirements by focusing on the how, rather than the fundamental what and who.

Thanks,
Dan Thompson
 
PREDATAR Control23

Dan,

What you are asking has been asked for a very long time in some way or another.

ISP (TSM) - since its roots came from ADSM on the mainframe - was very weak in reporting capabilities. As a technical guy it does not bother me that there are some shortcomings as I know there are ways to get this information. But as a management guy, I am drawn to look at alternatives in order to get reports easily for present and future decisions.

I love ISP and would hate to see its demise because of the lack of a flexible, and dynamic reporting capability. So here is a low down of things that I know of that has been asked for years.

1. Missed and failed backups - a not geeky narrative of what to look for to resolve the issue
2. A user driven reporting tool and not a canned reporting tool - let the user choose what to see dynamically like, see backup trends by node or group of nodes, make trend analysis on what totals would be.
3. Display reports by day, by week or by month as the user so chooses
4. The ability to display (again as the user see fit) common errors by node, node groups and have the ability to see past and current trends side-by-side
5. The ability to see ISP server loading at any given point in time either historically or currently. This aids in distributing backup loads by time and number of nodes.
6. The reports can be generated daily, weekly, or monthly as needed - this means that the ability exist to have reports generated at moments notice
7. The ability put multiple ISP server reports - performance matrix, node loading, etc. - side-by-side as needed.

All of the reports should have graphing and table outputs, and can be exported to PDF or HTML for printing or web publishing.

A long list - and I doubt if IBM can even address this in giving us a dynamic and flexible base reporting tool.
 
Last edited:
PREDATAR Control23

There are three purposes for reports: 1) Spot immediate problems, 2) Plan short-term everyday actions, such as creation of new nodes or adding scratch tapes, 3) Long-term trends analysis. A given report can reasonably only target one of these purposes at a time. But the need for all three is obvious.

With 3 production TSM servers plus a library manager server, we need, and get lots of reports. We produce both separate and combined reports. Nearly all reports combine information from several sources, from at least two separate SQL tables. (nodes, file spaces, and occupancy, for instance) A primary combined report is produced daily, and the parts I look at first include which nodes have suddenly grown, nodes that have missed backups for X days in a row, orphan tape volumes, how well collocated data really is collocated, and such typical stats. A report that I use frequently is a breakdown by collocation group, so I can easily put new nodes into a collocation group that makes sense. You must do this when nodes are initially registered, before the initial full backup, or else you will never have effective collocation groups. These reports are generated by IBM SPSS, partly for historic reasons (we're already licensed for it, and we've used it for a long time) and also it seems to be considerably more flexible than IBM Cognos. Plus, if you're so inclined, it has full statistical analysis.

Every morning at 0830, it runs DB2 SELECTs to extract the data it needs, and then combines it for faster subsequent analysis. This includes both the regular daily reports mentioned above, and also special one-time reports I might need for special conditions or projects.

We also use a free package called cricket (http://cricket.sourceforge.net/) to easily graph such things as sessions, amount of data in storage pools, fullness of database and log, number of free scratch tapes, network speed in and out of the servers, CPU utilization of the server machines, and a lot more. The cricket graphs can be displayed in any time span you want - immediate, weekly, and so on up to 5-years. Here's a sample cricket graph showing disk storage pools, which gives me an instant health check on how client backups are doing, and how migration is working.

mini-graph.cgi


These two packages, IBM SPSS and cricket, give us pretty much all the information we need, and they are very flexible to new circumstances, such as when we first adopted a Library Manager setup which was a major change.

While a set of pre-built reports could be useful, I would suggest only supplying them as samples to be modified to suit local needs. I've seen some 3rd party tools that produced beautiful color graphs, but that we couldn't really adapt to our local requirements. That made them useless. Flexibility and tailorability are key.

Roger Deschner, University of Illinois at Chicago
 
PREDATAR Control23

While there is currently the ability to send out backup event status reports for each node associated to a schedule. It would be nice to not have to use SQL select statements in order to achieve this via the OC to email out reports. This functionality should be something that is built in and produces a much nicer output in the report emails that are sent to recipients. This would be much more useful for Spectrum Protect servers that are being utilized by many different departments or even companies in a cloud env where that division and granularity of backup success / failure reporting is required.

Also, segregated vm backup reports. Do they even exist? Again without using SQL select statements and drilling down into the DC proxy node file space tables to get anything half respectable, there's simply nothing within the OC that provides anything close to a nice looking backup report for VM backups. SQl Select Statement outputs through the OC are pretty ugly looking at best.

Just seems reporting doesn't cater for segregation of data within the OC. It's very much ALL or NOTHING. Has this been addressed in SP+ ... ?

Thanks
 
PREDATAR Control23

Having attended the recent Tivoli/ISP User group meeting in London last month there is good news on the reporting front within the OC at v8.1.4.
No specific details mentioned but much improvement in this area is on the way. :)
ETA Mid-late December 2017
 
PREDATAR Control23

Not much to add but I'd like to parrot what moon-buddy, roger and ILCattivo have stated above. All very good points and much needed features/reports.
 
PREDATAR Control23

Having attended the recent Tivoli/ISP User group meeting in London last month there is good news on the reporting front within the OC at v8.1.4.
It was released last Friday. There's a demo on YouTube:
 
PREDATAR Control23

Having watched the demo that marclant has inserted into the previous post and after upgrading one of the OC's I use to 8.1.4 (Servers at the same level) there still appears to be some work required to reporting when a particular Protect server has multi-tenants.
Its deffo a move in the right direction but it just falls a little short of what I specifically was after to make life easier and more professional looking to the end customer/s.
 
PREDATAR Control23

I would like to see the following:

- Reports should be aimed at providing data to management and not at admins or technical staff- in other words, more non-technical staff friendly
- I would like to see report on client name, last backup, how much backed up
- A report on replication which shows Node Name, Backup Size, Replicated Size, Pending Replication, in progress? etc.

Yes there are reports available now, but they are not management friendly.
 
PREDATAR Control23

Hello,

1) What report(s) do you need?
2) What is the fundamental task you want the report to enable? (for example: I want to know how much of the planned capacity of my TSM/Protect server is being utilized, for any and all resources).
3) Does the fundamental task require information from multiple sources?
4) What is the scope of the report? Multiple servers, single server, single node, single schedule, etc.
5) What form do you want this report (pre-formatted like the daily OC report, tabular, chart, don't care)?
6) Who is the intended recipient (TSM/Protect admins, operations, mid-level mgmt, upper mgmt, etc.)?
7) What is the distribution method (email, post to a web server and email link, pdf, spreadsheet, etc.)?
8) How often is the report needed (daily, weekly, other)?
9) If you want to give an example of what you are describing, please feel free to share (SQL query, snapshot or mock-up).
Dan Thompson

1) Backup missed and failed , istance ,node name, schedule name , number of days since is failing/missing , possibility to exclude locked nodes or to use some filters
2)if backup by nodes are completed or not
3)yes
4) i need a possibility to adapt this report to different situation
5) Prefomatted , csv , unix and windows format
6) will be my choiche , who can create the report is who decide
7) email ,OVO zabbix and remedy ticket , file sent as attachement
8) daily
9)

select \
f.NODE_NAME, \
n.CONTACT, \
e.SCHEDULE_NAME, \
e.status, \
e.result, \
max(DAYS(current_timestamp) - DAYS(f.BACKUP_END)) as FAIL_SINCE \
from events e, filespaces f, nodes n \
where f.node_name=e.node_name \
and f.node_name=n.node_name \
and (e.status='Missed' or e.status='Failed') \
and n.LOCKED='NO' \
group by f.NODE_NAME,n.CONTACT, e.SCHEDULE_NAME,e.status,e.result \
having max(DAYS(current_timestamp) - DAYS(f.BACKUP_END))>1 \
order by f.NODE_NAME, e.SCHEDULE_NAME, FAIL_SINCE DESC;

and

select TO_CHAR(CHAR(scheduled_start),'YYYY-MM-DD HH24:MI:SS') as SCHEDULED_START,TO_CHAR(CHAR(actual_start),'YYYY-MM-DD HH24:MI:SS') as ACTUAL_START,TRANSLATE('a bc:de:fg', DIGITS(completed - actual_start), '_______abcdefgh_____',' ') as "ELAPTIME (D HHMMSS)", DOMAIN_NAME,SCHEDULE_NAME,NODE_NAME,STATUS,RESULT,(DAYS(current_timestamp) - DAYS(COMPLETED)) as LAST_COMPLETED,REASON from events where status!='Completed' and status!='Future' and status!='Progress' and SCHEDULED_START>current_timestamp-24 hours
 
Top