Re: RAIT in 2.4.3b4

Marc W. Mengel on Tue 25/03 17:40 -0600:
> I see one flaw in your script -- to wit, you're writing
> twice as much total data in your first loop as your
> second.
> 
> Your first part does three writes of 256MB in parallel
> for a total of 768MB.  So 768MB / 32sec == 24MB/sec.
> 
> Your second does one 3-stripe RAIT write of 256MB, which
> does 256MB of data plus 128M of checksum blocks for a
> total of 384Mb.  So 384MB / 16sec == 24MB/sec.

But the whole point of the test was to verify your assertion
that using some form of non-blocking IO in the RAIT code was
not necessary because tape buffering made it a non-issue;
each block of data was written into the drive's buffer
immediately anyways, so the system call returned while the
device wrote out the data to tape asynchronously.  By
comparing a triply-parallel write to a RAIT-enabled 3-tape
write, the times should have been basically the same if your
assertion was correct.

What happened from the script run that I posted was that the
RAIT code appeared to be wildly *faster* and more variable
than even the parallel writes, which I didn't understand at
all...

I did find a couple of bogons in my script that made it
useless though: I used the *same tape device* in the first
loop (which of course serialized the runs), and also the
version I had attached did not use the right syntax for the
RAIT device, which is I believe what caused the variation.

So posted at bottom is the fixed script, and here are the
new numbers:

    running parallel amdd tests
    now testing 524288 blocks of size 512...0:35.63 elapsed
    now testing 262144 blocks of size 1024...0:35.70 elapsed
    now testing 65536 blocks of size 4096...0:34.96 elapsed
    now testing 4096 blocks of size 65536...0:34.95 elapsed
    now testing 512 blocks of size 524288...0:34.87 elapsed
    now testing 64 blocks of size 4194304...0:34.90 elapsed

    running rait amdd tests
    now testing 524288 blocks of size 512...0:31.95 elapsed
    now testing 262144 blocks of size 1024...0:34.84 elapsed
    now testing 65536 blocks of size 4096...0:33.55 elapsed
    now testing 4096 blocks of size 65536...0:33.80 elapsed
    now testing 512 blocks of size 524288...0:33.58 elapsed
    now testing 64 blocks of size 4194304...0:34.17 elapsed

These results make much more sense and basically prove your
assertion.  I have done a little poking around and
discovered that my st driver does anynchronous writes on its
own as an option, and will also use do its own write
buffering in addition to the several meg buffer in my AIT-2
drives (this might explain the lack of variation when using
different block sizes; they're just coalesced by the driver)

So in short, you were 100% correct that, at least with my
AIT-2 drives and the Linux 2.4 st driver, there is no need
to implement async IO in the RAIT code.

-- 
Scott

#!/bin/bash

amroot=~/tmp/cvs/amdd-stress/amanda-2.4.4
amldpath=$amroot/tape-src/.libs:$amroot/common-src/.libs
dd="LD_LIBRARY_PATH=$amldpath $amroot/tape-src/.libs/amdd"

mbytes=256
blksizes=(512 1024 4096 65536 524288 4194304)

echo each run is $mbytes megabytes

for n in 0 1 2; do mt -f /dev/nst$n rewind; done

echo
echo running parallel amdd tests
for blksz in ${blksizes[@]}; do
        cnt=$(($(($mbytes << 20)) / $blksz))
        echo -n now testing $cnt blocks of size $blksz...
        /usr/bin/time -f %E\ elapsed sh -c "
                $dd if=/dev/zero of=/dev/nst0 bs=$blksz count=$cnt 2>/dev/null &
                $dd if=/dev/zero of=/dev/nst1 bs=$blksz count=$cnt 2>/dev/null &
                $dd if=/dev/zero of=/dev/nst2 bs=$blksz count=$cnt 2>/dev/null
        "
        sleep 1 # seem to need this or I get dev busy errors
done

for n in 0 1 2; do mt -f /dev/nst$n rewind; done

echo
echo running rait amdd tests
for blksz in ${blksizes[@]}; do
        cnt=$(($(($mbytes << 20)) / $blksz))
        echo -n now testing $cnt blocks of size $blksz...
        /usr/bin/time -f %E\ elapsed sh -c \
                "$dd if=/dev/zero of=rait:/dev/nst{0,1,2} bs=$blksz \
                 count=$cnt 2>/dev/null"
        sleep 1
done