Thanks for the clue, here is the problem:
# set | grep DSMI_LOG
DSMI_LOG=/home/db2inst1/dsmilog.log
# file /home/db2inst1/dsmilog.log
dsmilog.log: ascii text

This should be a directory not a file! My mistake :(

Another problem appeared:
# db2 backup db mydb use tsm
tsm: TSMSERVER> q ac
..
ANR0422W Session 86 for node AA (DB2/6000) refused - node name not
registered.

I registered node "AA" and db2 backup succeded this time.
My question is that what this AA node is and how I can rename it?
"q node AA f=d" prints this information:
Node Name: AA
Platform: DB2/6000
Client OS Level: 6.1
...
TCP/IP Name: testlpar
...

Regards,
Mehdi Salehi