Martin,
Thanks for the details, particularly the exist status list for rsync.
More questions:
1. When you say “restart rsync” that means start a new, non-incremental, rsync, right? Or do you think it should it be possible to attempt another incremental rsync?
2. Any idea if we can recover from the error message I sent over? !FATAL: logger_load: BBPrename to sql_snapshots_bid failed
3. Is stopping and locking MonetDB sufficient to ensure a consistent state? Does stopping allow running queries to complete, or at least modifications complete? Or are the sys.sessions query and sys.shutdown also required?
Thanks,
Vince
On 1/24/17, 11:11 AM, "users-list on behalf of Martin Kersten" wrote:
Hi Vincent
On 24/01/2017 17:51, Vincent Sheffer wrote:
> I have been attempting to create a viable period backup strategy for MonetDB and have had little success to date.
>
> Our database is way too big to use msqldump, so our only option is the one outlined here: https://www.monetdb.org/Documentation/UserGuide/FastDumpRestore.
>
> I follow those steps and use rsync to do an incremental backup of the dbfarm to another device (from fast SSD to RAID5 array). That works fine, until it doesn’t, in which case the back up gets corrupted somehow and MonetDB won’t start from the backup. That is the state I am currently in.
>
Key issue is that indeed the database server has been stopped before you start a rsync.
Furthermore, Rsync is an elaborate program with lots of options.
If an rsync fails, for whatever system error, indeed the backup data can not be trusted.
The list of possible errors to cope with are:
1 Syntax or usage error
2 Protocol incompatibility
3 Errors selecting input/output files, dirs
4 Requested action not supported: an attempt was made to manipulate 64-bit files on a platform that cannot support them; or an option was specified that is supported by the client and not by the server.
5 Error starting client-server protocol
6 Daemon unable to append to log-file
10 Error in socket I/O
11 Error in file I/O
12 Error in rsync protocol data stream
13 Errors with program diagnostics
14 Error in IPC code
20 Received SIGUSR1 or SIGINT
21 Some error returned by waitpid()
22 Error allocating core memory buffers
23 Partial transfer due to error
24 Partial transfer due to vanished source files
25 The --max-delete limit stopped deletions
30 Timeout in data send/receive
35 Timeout waiting for daemon connection
From all these exit values, the backup version can not be trusted and Rsync should be restarted.
Recurring errors can indicate a broken disk.
Recovering from a corrupted backup (disk) can not be detected by a DBMS.
regards, Martin
> This is the error that I am getting:
>
> !FATAL: logger_load: BBPrename to sql_snapshots_bid failed
>
> Any help on
>
> 1. Fixing the existing backup, and/or
> 2. Helping me understand a better way to do period, incremental backups.
>
> Thanks,
> Vince
>
>
> The information transmitted, including any attachments, is intended only for the individual or entity to which it is addressed, and may contain confidential and/or privileged information. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by individuals or entities other than the intended recipient is prohibited, and all liability arising therefrom is disclaimed. If you have received this communication in error, please delete the information from any computer and notify the sender.
> _______________________________________________
> users-list mailing list
> users-list@monetdb.org
> https://www.monetdb.org/mailman/listinfo/users-list
>
_______________________________________________
users-list mailing list
users-list@monetdb.org
https://www.monetdb.org/mailman/listinfo/users-list
