Richard Scobie wrote:
> I have a 2.8GHz Core i7 machine backing up uncompressable data spooled 
> onto an 8 drive RAID5, to LTO-4 tape.
> 
> Our requirements now dictate that data encryption must be used on the 
> tapes and having configured this, it seems that one core is saturated 
> encrypting the data and the result is that tape write speed is now about 
> 50% slower than when encryption is not used.

The process saturating a core is the file daemon, right?


> Would it be possible to optimise this task by perhaps reading data in 
> "chunks", which in turn can be encrypted by a core each, before being 
> recombined and written out to tape?


Bacula uses OpenSSL for crypto support. It doesn't seem to support any
other crypto libraries like NSS or GnuTLS.

OpenSSL supports hardware crypto acceleration for some cyphers in a
largely transparent manner. This is one option. If Bacula doesn't "just
work" with hardware crypto I'd expect it to be a one-line patch to add
support, going by what I've had to do to enable it in other software.

Some hardware, like the Via C7 series of CPUs, have built-in AES crypto
hardware (PadLock) that on a single thread can do *insane* encryption
rates. On the older C3 series CPUs I've had no problems saturating a
100MBit line with encrypted ssh data, despite the gutless 400MHz C3 CPU.

Intel has introduced similar instructions on their Xeon 5600 series:
 
http://software.intel.com/en-us/articles/boosting-openssl-aes-encryption-with-intel-ipp/


I'd be lovely to be able to use the IPP libraries in Bacula (and many
other things) for parallel crypto and many other parallel tasks, as
they're excellent even without special hardware. Unfortunately they're
rather GPL-incompatible and are only "free" for non-commercial use.

( The Intel Thread Building Blocks library *is* open source under GPLv2,
though, and if Bacula wasn't already using pthreads directly would be
rather nice: http://www.threadingbuildingblocks.org/ )


Anyway, if you need a software-only option, it's necessary to:

1) get OpenSSL to use multiple cores for encryption internally;
2) get Bacula to use OpenSSL to encrypt blocks using worker
   threads using a suitable block cypher; or
3) Use another crypto library that automatically parallelizes.


None of these look easy by any stretch. (2) is probably most realistic,
but as OpenSSL does some internal locking and serialization it may not
be possible to encrypt on multiple threads even when using a simple
block cypher where one block doesn't depend on the next or previous. I
don't know much about OpenSSL and can't say more without a lot more
digging. For all I know it might be necessary to ask OpenSSL for the
session key, then use its low level crypto functions to encrypt blocks
rather than using the higher-level stream/session interface.

While not directly OpenSSL related, this might also be of interest:

  http://www.psc.edu/networking/projects/hpn-ssh/
  http://www.psc.edu/networking/projects/hpn-ssh/papers/a14-rapier.pdf

It doesn't touch on OpenSSL, but at least it's using a highly parallel
AES cypher...






> I'd use the hardware encryption (which presumably has no performance 
> impact), that is an option on this autochanger, except they want $2500 
> for it...

Probably because it has a custom ASIC for the crypto algorithm in use to
allow it to go fast enough.

The trouble with this is that if your tape drive/changer dies, you
generally need another one with the same hardware crypto to restore.
This is a really, really ugly situation for disaster recovery.

( Maybe things have improved since I switched away from tape, and now
the hardware crypto is just an accelerator and you can still load your
keys for driver-based crypto instead. I doubt it, though. )

--
Craig Ringer

------------------------------------------------------------------------------
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users