Any more ideas why this error appears sometimes?

--
Silver

On 25.01.2016 10:42, Silver Salonen wrote:

Yeah, so what can we conclude from this now? :)

--
Silver

On 22.01.2016 17:55, Wanderlei Huttel wrote:

Hi Silver

I've made a test using your script and worked fine!

I'm using bacula 7.4.0.

- MySQL user has all privileges;
grant all on *.* to 'user'@'localhost' identified by 'password';

- The directory and file /mnt/fifo/intranet.sql doesn't exists
- Permission of script is 775

##################################################

# FileSet teste bpipe

FileSet {

  Name = "FileSet_Bpipe_Bacula"

  Include {

     Options {

        signature = md5

        compression = gzip

     }

  Plugin = "bpipe:/backup/dump_mysql.sql:/usr/local/bin/backup_mysql.sh intranet:/usr/local/bin/backup_mysql.sh -R"

  }

}

##################################################

Backup Log

2016-01-22 13:43:01   bacula-dir JobId 3012: Start Backup JobId 3012, Job=Backup_Bpipe_Bacula.2016-01-22_13.42.59_48
2016-01-22 13:43:01   bacula-dir JobId 3012: Using Device "DiscoLocal1" to write.
2016-01-22 13:43:01   bacula-sd JobId 3012: Volume "Volume-Default-0001" previously written, moving to end of data.
2016-01-22 13:43:01   bacula-sd JobId 3012: Ready to append to end of Volume "Volume-Default-0001" size=35,757,662
2016-01-22 13:43:05   bacula-sd JobId 3012: Elapsed time=00:00:04, Transfer rate=1.680 M Bytes/second
2016-01-22 13:43:05   bacula-sd JobId 3012: Sending spooled attrs to the Director. Despooling 285 bytes ...
2016-01-22 13:43:05   bacula-dir JobId 3012: Bacula bacula-dir 7.4.0 (16Jan16):
  Build OS:               x86_64-unknown-linux-gnu debian 8.2
  JobId:                  3012
  Job:                    Backup_Bpipe_Bacula.2016-01-22_13.42.59_48
  Backup Level:           Full
  Client:                 "srv_bacula-fd" 7.4.0 (16Jan16) x86_64-unknown-linux-gnu,debian,8.2
  FileSet:                "FileSet_Bpipe_Bacula" 2016-01-22 13:42:59
  Pool:                   "Pool-Default" (From Job resource)
  Catalog:                "MyCatalog" (From Client resource)
  Storage:                "StorageLocal1" (From Pool resource)
  Scheduled time:         22-Jan-2016 13:42:59
  Start time:             22-Jan-2016 13:43:01
  End time:               22-Jan-2016 13:43:05
  Elapsed time:           4 secs
  Priority:               10
  FD Files Written:       1
  SD Files Written:       1
  FD Bytes Written:       6,722,331 (6.722 MB)
  SD Bytes Written:       6,722,542 (6.722 MB)
  Rate:                   1680.6 KB/s
  Software Compression:   85.1% 6.7:1
  Snapshot/VSS:           no
  Encryption:             no
  Accurate:               no
  Volume name(s):         Volume-Default-0001
  Volume Session Id:      7
  Volume Session Time:    1453470218
  Last Volume Bytes:      42,492,736 (42.49 MB)
  Non-fatal FD errors:    0
  SD Errors:              0
  FD termination status:  OK
  SD termination status:  OK
  Termination:            Backup OK
2016-01-22 13:43:05   bacula-dir JobId 3012: Begin pruning Jobs older than 1 year .
2016-01-22 13:43:05   bacula-dir JobId 3012: No Jobs found to prune.
2016-01-22 13:43:05   bacula-dir JobId 3012: Begin pruning Files.
2016-01-22 13:43:05   bacula-dir JobId 3012: No Files found to prune.
2016-01-22 13:43:05   bacula-dir JobId 3012: End auto prune.
2016-01-22 13:43:05   bacula-dir JobId 3012: shell command: run AfterJob "/etc/bacula/scripts/send_whatsapp.sh "Client=srv_bacula-fd" "JobId=3012" "JobName=Backup_Bpipe_Bacula" "JobBytes=6722331" "JobFiles=1" "Level=Full" "Pool=Pool-Default" "Storage=StorageLocal1" "Status=OK""
2016-01-22 13:43:06   bacula-dir JobId 3012: AfterJob: INFO:yowsup.demos.sendclient.layer:Message sent

Restore Log

2016-01-22 13:44:44   bacula-dir JobId 3013: Start Restore Job Restore_Files.2016-01-22_13.44.42_56
2016-01-22 13:44:44   bacula-dir JobId 3013: Using Device "DiscoLocal1" to read.
2016-01-22 13:44:44   bacula-sd JobId 3013: Ready to read from volume "Volume-Default-0001" on file device "DiscoLocal1" (/mnt/disco_usb).
2016-01-22 13:44:44   bacula-sd JobId 3013: Forward spacing Volume "Volume-Default-0001" to file:block 0:35757662.
2016-01-22 13:44:45   bacula-sd JobId 3013: End of Volume at file 0 on device "DiscoLocal1" (/mnt/disco_usb), Volume "Volume-Default-0001"
2016-01-22 13:44:45   bacula-sd JobId 3013: End of all volumes.
2016-01-22 13:44:45   bacula-sd JobId 3013: Elapsed time=00:00:01, Transfer rate=6.722 M Bytes/second
2016-01-22 13:45:57   bacula-dir JobId 3013: Bacula bacula-dir 7.4.0 (16Jan16):
  Build OS:               x86_64-unknown-linux-gnu debian 8.2
  JobId:                  3013
  Job:                    Restore_Files.2016-01-22_13.44.42_56
  Restore Client:         srv_bacula-fd
  Start time:             22-Jan-2016 13:44:44
  End time:               22-Jan-2016 13:45:57
  Files Expected:         1
  Files Restored:         1
  Bytes Restored:         45,185,460
  Rate:                   619.0 KB/s
  FD Errors:              0
  FD termination status:  OK
  SD termination status:  OK
  Termination:            Restore OK
2016-01-22 13:45:57   bacula-dir JobId 3013: Begin pruning Jobs older than 1 year .
2016-01-22 13:45:57   bacula-dir JobId 3013: No Jobs found to prune.
2016-01-22 13:45:57   bacula-dir JobId 3013: Begin pruning Files.
2016-01-22 13:45:57   bacula-dir JobId 3013: No Files found to prune.
2016-01-22 13:45:57   bacula-dir JobId 3013: End auto prune.

Best Regards

Wanderlei

2016-01-22 11:21 GMT-02:00 Silver Salonen <silver.salonen AT gmail DOT com>:

Debug output concerning mysqldump from /opt/bacula/bin/bacula-fd -d 200 -c bacula-fd.conf -f

job.c:1227-54555 P bpipe:/srv/mysqldump.sql:/bin/sh /etc/bacula/scripts/mysqldump.sh mydb:/etc/bacula/scripts/mysqldump.sh -R
fd_plugins.c:329-54555 plugin cmd=bpipe:/srv/mysqldump.sql:/bin/sh /etc/bacula/scripts/mysqldump.sh mydb:/etc/bacula/scripts/mysqldump.sh -R
fd_plugins.c:127-54555 name=bpipe:/srv/mysqldump.sql:/bin/sh /etc/bacula/scripts/mysqldump.sh mydb:/etc/bacula/scripts/mysqldump.sh -R len=5 plugin=bpipe-fd.so plen=5
bpipe-fd.c:215-54555 bpipe-fd: PluginCommand=bpipe:/srv/mysqldump.sql:/bin/sh /etc/bacula/scripts/mysqldump.sh mydb:/etc/bacula/scripts/mysqldump.sh -R
accurate.c:230-54555 add fname=</mysql/mysqldump.sql> lstat=A A IHA A A A A -B BAA B BWoVsJ BWoVsJ BWoVsJ A A E  delta_seq=0 chksum=
fd_plugins.c:329-54555 plugin cmd=bpipe:/srv/mysqldump.sql:/bin/sh /etc/bacula/scripts/mysqldump.sh mydb:/etc/bacula/scripts/mysqldump.sh -R
fd_plugins.c:505-54555 plugin=bpipe-fd.so plen=5 cmd=bpipe:/srv/mysqldump.sql:/bin/sh /etc/bacula/scripts/mysqldump.sh mydb:/etc/bacula/scripts/mysqldump.sh -R len=5
fd_plugins.c:127-54555 name=bpipe:/srv/mysqldump.sql:/bin/sh /etc/bacula/scripts/mysqldump.sh mydb:/etc/bacula/scripts/mysqldump.sh -R len=5 plugin=bpipe-fd.so plen=5
fd_plugins.c:521-54555 Command plugin = bpipe:/srv/mysqldump.sql:/bin/sh /etc/bacula/scripts/mysqldump.sh mydb:/etc/bacula/scripts/mysqldump.sh -R
bpipe-fd.c:253-54555 bpipe-fd: pluginEvent cmd=bpipe:/srv/mysqldump.sql:/bin/sh /etc/bacula/scripts/mysqldump.sh mydb:/etc/bacula/scripts/mysqldump.sh -R
fd_plugins.c:583-54555 startBackup returned type=3, fname=/srv/mysqldump.sql
backup.c:289-54555 FT_REG saving: /srv/mysqldump.sql
backup.c:390-54555 bfiled: sending /srv/mysqldump.sql to stored
fd_plugins.c:778-54555 send_plugin_name=bpipe:/srv/mysqldump.sql:/bin/sh /etc/bacula/scripts/mysqldump.sh mydb:/etc/bacula/scripts/mysqldump.sh -R
fd_plugins.c:802-54555 send plugin start/end: 1 1 1 bpipe:/srv/mysqldump.sql:/bin/sh /etc/bacula/scripts/mysqldump.sh mydb:/etc/bacula/scripts/mysqldump.sh -R
backup.c:1352-54555 No strip for /srv/mysqldump.sql
bpipe-fd.c:360-54555 bpipe-fd: IO_OPEN fd=7f33e4003528 reader=/bin/sh /etc/bacula/scripts/mysqldump.sh mydb
fd_plugins.c:778-54555 send_plugin_name=bpipe:/srv/mysqldump.sql:/bin/sh /etc/bacula/scripts/mysqldump.sh mydb:/etc/bacula/scripts/mysqldump.sh -R
accurate.c:46-54555 </srv/mysqldump.sql> not found to be marked as seen
accurate.c:153-54555 deleted fname=/mysql/mysqldump.sql seen=0

As accurate.c complained about the file I disabled Accurate backup, but the warning remains:

JobId 54556: Warning: Cannot stat file /srv/mysqldump.sql: ERR=No such file or directory

--
Silver

On 22.01.2016 13:45, Silver Salonen wrote:

I know how bpipe works and my method using script is to work around an issue with bpipe erroring out with "Error closing stream for pseudo file" (no idea why, but running the same mysqldump command from inside script solves this). Contents of the script is really straight-forward and doesn't change behavior of the dump being sent to STDOUT:

#!/bin/sh

 

user=backup

pswd=PASSWORD

 

if [ "$1" = "-R" ]; then

    /usr/bin/mysql --default-character-set=utf8 -u "$user" -p"$pswd"

else

    /usr/bin/mysqldump --single-transaction --routines  --events --opt --max_allowed_packet 128M --default-character-set=utf8 --log-error=/var/log/mysql/mysqldump_err.log -u "$user" -p"$pswd" --databases $*

fi

 

exit 0

BTW, I discovered that the error "bpipe-fd: Error closing for file /srv/mysqldump.sql" was caused of the wrong credentials I had in the script. The first error still remains.

--
Silver

On 22.01.2016 12:55, Heitor Faria wrote:

Hello Silver,

Hello Silver [2],

Please read: http://bacula.us/using-bpipe-to-stream-dumps-vm-clones-and-another-data-to-your-backup

bpipe reads a command data that is directed to stdout and stores in backup with the name you put in the 2nd variable (fifo path) => I hope is what you are trying to achieve with your scripts. While in restore Bacula uses the last bpipe configuration variable (command) to read from stdin and restore your data with the application of your choice. I never saw a bpipe usage documentation with scripts, but with commands directly. E.g.: Plugin = “bpipe:/mnt/fifo/heitorfaria.sql:mysqldump -u root -ppasswd –databases heitorfaria:mysql -u root -ppasswd″

The beauty of bpipe is you don't need local disk space to generate the dumps since fifos actually don't hold data (it just streams it to an end) and you may also automate application restore directly to it.

I recorded some videoclasses that you can find in the same URL post I provided.  

JobId 54542: Error: /srv/mysqldump.sql mtime changed during backup.
JobId 54542: Error: bpipe-fd: Error closing for file /srv/mysqldump.sql: 268435583
...
  Termination:            Backup OK -- with warnings

--
Silver

On 22.01.2016 10:56, Ana Emília M. Arruda wrote:

Hello Silver,

The same error? Have you checked permissions?

Best regards,

Ana

On Fri, Jan 22, 2016 at 9:19 AM, Silver Salonen <silver.salonen AT gmail DOT com> wrote:

After creating those folders, those errors still appeared. Any other ideas?

--
Silver

On 21.01.2016 13:43, Silver Salonen wrote:

Aha, I indeed didn't have /mysql directory. Thanks a bunch! :)

PS. Script permissions are correct as otherwise the dump wouldn't have been created.

--
Silver

On 21.01.2016 12:21, Heitor Faria wrote:

Hello Silver,

Does the /mysql directory exist? It must.
Make sure also bacula-fd daemon OS user has proper permissions to run the scripts

Regards,
===========================================================================
Heitor Medrado de Faria - LPIC-III | ITIL-F |  Bacula Systems Certified Administrator II
Do you need Bacula training? http://bacula.us/video-classes/
+55 61 8268-4220
Site: http://bacula.us FB: heitor.faria
===========================================================================

Enviado por TypeApp

Em 21 de jan de 2016, pelo 06:37, Silver Salonen <silver.salonen AT gmail DOT com> escrito:

Hi.

I'm backing up MySQL databases from several servers with Fileset line:
     Plugin = 
"bpipe:/mysql/mysqldump.sql:/etc/bacula/scripts/mysqldump.sh 
mydb:/etc/bacula/scripts/mysqldump.sh -R"

FD on some of those servers give the following error:
     Warning: Cannot stat file /mysql/mysqldump.sql: ERR=No such file or 
directory

I've confirmed that actually the MySQL database is still backed up and 
can be restored too. So why the error? Anybody knows or can guess?

--
Silver

------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users