Bacula-users

Re: [Bacula-users] ssd drive question - Throttle clients

2012-06-07 13:05:34
Subject: Re: [Bacula-users] ssd drive question - Throttle clients
From: Tim Krieger <tim.krieger AT neverblue DOT com>
To: 'Josh Fisher' <jfisher AT pvct DOT com>, "bacula-users AT lists.sourceforge DOT net" <bacula-users AT lists.sourceforge DOT net>
Date: Thu, 7 Jun 2012 10:03:00 -0700
Kind of branching off here but felt I should toss in my 2 cents re: Throttling 
clients

We have a similar problem where some remote clients have low bandwidth pipes 
and we can't consume all available bandwidth.  Our environment is mostly linux 
so and we achieve throttling using baculas client run before and client run 
after options and some simple firewall traffic shaping scripts.

Mac os may have the same/similar functionality available, if not, you can 
always use the scripts server side to achieve the same(assuming a linux storage 
server) just ensure some careful management and tweaking to prevent collisions.

Scripts :

<ADD LIMIT>
#!/bin/bash
## rate limit bacula transfer
## /opt/bacula/bin/add_rate_limit.sh

BANDWIDTH="1gbit"
RATE_LIMIT="10mbit"
DEST="bacula.server.org"
IF="eth1"

LIMITED=`tc -d class ls dev $IF | wc -l`

if [ $LIMITED -lt 2 ]
then
  /sbin/tc qdisc add dev $IF root handle 1:0 htb default 10
  /sbin/tc class add dev $IF parent 1:0 classid 1:1 htb rate $BANDWIDTH
  /sbin/tc class add dev $IF parent 1:1 classid 1:10 htb rate $BANDWIDTH
  /sbin/tc class add dev $IF parent 1:1 classid 1:11 htb rate $RATE_LIMIT
  /sbin/iptables -I OUTPUT -t mangle -d $DEST -p tcp --dport 9103 -j MARK 
--set-mark 11
  /sbin/tc filter add dev $IF parent 1:0 prio 1 protocol ip handle 11 fw flowid 
1:11
else
  echo "Already have rate limits in place"
fi
</ADD LIMIT>

<REMOVE LIMIT>
#!/bin/bash
## Remove rate limit
## /opt/bacula/bin/remove_rate_limit.sh

DEST="bacula.server.org"
IF="eth1"

/sbin/iptables -D OUTPUT -t mangle -d $DEST -p tcp --dport 9103 -j MARK 
--set-mark 11
/sbin/tc qdisc del dev $IF root
</REMOVE LIMIT>





-----Original Message-----
From: Josh Fisher [mailto:jfisher AT pvct DOT com]
Sent: Monday, June 04, 2012 8:04 AM
To: bacula-users AT lists.sourceforge DOT net
Subject: Re: [Bacula-users] ssd drive question


On 5/31/2012 9:19 PM, Randy Katz wrote:
> Hi,
>
> Has anyone tried to use an SSD drive to house the MySQL database for
> Bacula and understand the
> performance differences, if any? I am wondering if it will provide a
> significant speed boost to running it
> on disk or not. Any clues might be very helpful, thanks.

Increased speed of what, exactly?

I moved a bacula db from a RAID1 array of two WD RE-4 drives to an old
80 GB Intel X25-M, which is not one of the fastest compared to more
recent SSDs. However it has random read and write performance that far
exceeds hard drives. Whether or not it constitutes a significant speed
boost for Bacula depends on whether or not MySQL was the bottleneck to
begin with. In my case, catalog backup with mysqldump was significantly
faster, file selection during a restore was more than twice as fast,
pruning/purging volumes was much faster. However, there was only a
modest improvement in backup job times. I turned off attribute spooling
and it didn't much affect backup times. I have thought about moving the
data spooling to SSD, but I suspect it would make little improvement.

In my case, I believe the clients themselves are the bottleneck. In
fact, believe it or not, I wish there were a way to throttle backup
speeds for particular clients. I have people that drop their network
connection on Macbook Pros because they get hot and noisy (from the
increased fan speed) when a backup is in progress, yet they take it home
with them at night. It's getting hard to get a backup of those machines.



------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users

This electronic mail transmission and any accompanying attachments contain 
confidential information intended only for the use of the individual or entity 
named above. Any dissemination, distribution, copying or action taken in 
reliance on the contents of this communication by anyone other than the 
intended recipient is strictly prohibited. If you have received this 
communication in error please immediately delete the e-mail and either notify 
the sender at the above e-mail address or by telephone at +1 250.386.5323.

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users