Hello, I hope anyone can help with database crash. In .gdk_lock is following: USR=1001 PID=6751 TIME=Mon Feb 10 16:04:17 2014 @ LOGON USR=1001 PID=6751 TIME=Mon Feb 10 16:04:18 2014 @ !FATAL: logger_new: there is a logger catalog, but no log file. Are you sure you are using the correct combination of database (--dbpath) and log directory (--set sql_logdir)? If you have done a recent update of the server, it may be that your logs are in an old location. You should then either use --set sql_logdir=<path to old log directory> or move the old log directory to the new location (/u01/ceosdata/database/monetdb/drmax/crm/zkdev1/sql_logs/sql/). USR=1001 PID=6751 TIME=Mon Feb 10 16:04:18 2014 @ LOGOFF Anyway to start the database without logs ? Thanks, Radovan
Hello It is not a crash as such, but a safeguard against an incorrect environment. A database logger should never be removed, because it is needed to realise transaction properties. The log message indicates what to do. regards, Martin On 10/02/14 16:08, Radovan Bičiště wrote:
Hello, I hope anyone can help with database crash. In .gdk_lock is following: USR=1001 PID=6751 TIME=Mon Feb 10 16:04:17 2014 @ LOGON USR=1001 PID=6751 TIME=Mon Feb 10 16:04:18 2014 @ !FATAL: logger_new: there is a logger catalog, but no log file. Are you sure you are using the correct combination of database (--dbpath) and log directory (--set sql_logdir)? If you have done a recent update of the server, it may be that your logs are in an old location. You should then either use --set sql_logdir=<path to old log directory> or move the old log directory to the new location (/u01/ceosdata/database/monetdb/drmax/crm/zkdev1/sql_logs/sql/). USR=1001 PID=6751 TIME=Mon Feb 10 16:04:18 2014 @ LOGOFF
Anyway to start the database without logs ? Thanks, Radovan
_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
Hello, I have recovered from a backup of VM and loaded missing data. My scenario is now: 1. run SELECT statement that I need to stop. 2. did stop and start database. 3. database is refusing to start with: ceos@lux-analytics:~$ monetdb start zkdev1 starting database 'zkdev1'... FAILED start: starting 'zkdev1' failed: database 'zkdev1' has crashed after starting, manual intervention needed, check monetdbd's logfile for details in .gdk_lock is: ceos@lux-analytics:/u01/ceosdata/database/monetdb/drmax/crm/zkdev1$ tail -f .gdk_lock USR=1001 PID=2583 TIME=Tue Feb 11 11:13:44 2014 @ LOGON Any suggestations? Thanks, Radovan On 02/10/2014 05:25 PM, Martin Kersten wrote:
Hello
It is not a crash as such, but a safeguard against an incorrect environment. A database logger should never be removed, because it is needed to realise transaction properties.
The log message indicates what to do.
regards, Martin
On 10/02/14 16:08, Radovan Bičiště wrote:
Hello, I hope anyone can help with database crash. In .gdk_lock is following: USR=1001 PID=6751 TIME=Mon Feb 10 16:04:17 2014 @ LOGON USR=1001 PID=6751 TIME=Mon Feb 10 16:04:18 2014 @ !FATAL: logger_new: there is a logger catalog, but no log file. Are you sure you are using the correct combination of database (--dbpath) and log directory (--set sql_logdir)? If you have done a recent update of the server, it may be that your logs are in an old location. You should then either use --set sql_logdir=<path to old log directory> or move the old log directory to the new location (/u01/ceosdata/database/monetdb/drmax/crm/zkdev1/sql_logs/sql/). USR=1001 PID=6751 TIME=Mon Feb 10 16:04:18 2014 @ LOGOFF
Anyway to start the database without logs ? Thanks, Radovan
_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
-- __________________________ Radovan Bičiště ceos data s.r.o. Pouchovská 153 500 03 Hradec Králové Czech Republic mobil CZ: +420 601 563 014 skype: rbiciste
On 11/02/14 11:17, Radovan Bičiště wrote:
Hello, I have recovered from a backup of VM and loaded missing data. What kind of VM are you using?
My scenario is now: 1. run SELECT statement that I need to stop. 2. did stop and start database. 3. database is refusing to start with: ceos@lux-analytics:~$ monetdb start zkdev1 starting database 'zkdev1'... FAILED start: starting 'zkdev1' failed: database 'zkdev1' has crashed after starting, manual intervention needed, check monetdbd's logfile for details
in .gdk_lock is: ceos@lux-analytics:/u01/ceosdata/database/monetdb/drmax/crm/zkdev1$ tail -f .gdk_lock USR=1001 PID=2583 TIME=Tue Feb 11 11:13:44 2014 @ LOGON The tail of the merovingian.log file is needed to assess why the system stopped. That keeps the server status errors.
regards, Martin
Any suggestations?
Thanks, Radovan
On 02/10/2014 05:25 PM, Martin Kersten wrote:
Hello
It is not a crash as such, but a safeguard against an incorrect environment. A database logger should never be removed, because it is needed to realise transaction properties.
The log message indicates what to do.
regards, Martin
On 10/02/14 16:08, Radovan Bičiště wrote:
Hello, I hope anyone can help with database crash. In .gdk_lock is following: USR=1001 PID=6751 TIME=Mon Feb 10 16:04:17 2014 @ LOGON USR=1001 PID=6751 TIME=Mon Feb 10 16:04:18 2014 @ !FATAL: logger_new: there is a logger catalog, but no log file. Are you sure you are using the correct combination of database (--dbpath) and log directory (--set sql_logdir)? If you have done a recent update of the server, it may be that your logs are in an old location. You should then either use --set sql_logdir=<path to old log directory> or move the old log directory to the new location (/u01/ceosdata/database/monetdb/drmax/crm/zkdev1/sql_logs/sql/). USR=1001 PID=6751 TIME=Mon Feb 10 16:04:18 2014 @ LOGOFF
Anyway to start the database without logs ? Thanks, Radovan
_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
On 02/11/2014 11:36 AM, Martin Kersten wrote:
On 11/02/14 11:17, Radovan Bičiště wrote:
Hello, I have recovered from a backup of VM and loaded missing data. What kind of VM are you using? We are using VMWare 5.1.
My scenario is now: 1. run SELECT statement that I need to stop. 2. did stop and start database. 3. database is refusing to start with: ceos@lux-analytics:~$ monetdb start zkdev1 starting database 'zkdev1'... FAILED start: starting 'zkdev1' failed: database 'zkdev1' has crashed after starting, manual intervention needed, check monetdbd's logfile for details
in .gdk_lock is: ceos@lux-analytics:/u01/ceosdata/database/monetdb/drmax/crm/zkdev1$ tail -f .gdk_lock USR=1001 PID=2583 TIME=Tue Feb 11 11:13:44 2014 @ LOGON The tail of the merovingian.log file is needed to assess why the system stopped. That keeps the server status errors.
I forgot to fill out important data: ceos@lux-analytics:/u01/ceosdata/database/monetdb/drmax/crm$ monetdbd version MonetDB Database Server v1.7 (Feb2013-SP3) Log from merovingian.log: 2014-02-11 11:35:46 MSG merovingian[1504]: Merovingian 1.7 (Feb2013-SP3) starting 2014-02-11 11:35:46 MSG merovingian[1504]: monitoring dbfarm /u01/ceosdata/database/monetdb/drmax/crm 2014-02-11 11:35:46 MSG merovingian[1504]: accepting connections on TCP socket 0.0.0.0:50000 2014-02-11 11:35:46 MSG merovingian[1504]: accepting connections on UNIX domain socket /tmp/.s.monetdb.50000 2014-02-11 11:35:46 MSG discovery[1504]: listening for UDP messages on 0.0.0.0:50000 2014-02-11 11:35:46 MSG control[1504]: accepting connections on UNIX domain socket /tmp/.s.merovingian.50000 2014-02-11 11:35:46 MSG discovery[1504]: new neighbour lux-analytics (lux-analytics-dev.internal.ceosdata.com) 2014-02-11 11:35:48 MSG discovery[1504]: new database mapi:monetdb://lux-analytics:50000/clf1 (ttl=660s) 2014-02-11 11:35:48 MSG discovery[1504]: new database mapi:monetdb://lux-analytics:50000/zkdev1 (ttl=660s) 2014-02-11 11:36:00 MSG control[1504]: (local): served status list 2014-02-11 11:36:00 MSG merovingian[1504]: database 'zkdev1' has crashed after start on 2014-02-11 11:28:43, attempting restart, up min/avg/max: 4s/22h/4w, crash average: 1.00 1.00 0.90 (250-133=117) 2014-02-11 11:36:01 MSG zkdev1[1509]: arguments: /usr/bin/mserver5 --dbpath=/u01/ceosdata/database/monetdb/drmax/crm/zkdev1 --set merovingian_uri=mapi:monetdb://lux-analytics:50000/zkdev1 --set mapi_open=false --set mapi_port=0 --set mapi_usock=/u01/ceosdata/database/monetdb/drmax/crm/zkdev1/.mapi.sock --set monet_vault_key=/u01/ceosdata/database/monetdb/drmax/crm/zkdev1/.vaultkey --set gdk_nr_threads=4 --set max_clients=64 --set sql_optimizer=default_pipe --set monet_daemon=yes 2014-02-11 11:36:12 MSG merovingian[1504]: database 'zkdev1' (1509) was killed by signal SIGSEGV 2014-02-11 11:36:12 ERR control[1504]: (local): failed to fork mserver: database 'zkdev1' has crashed after starting, manual intervention needed, check monetdbd's logfile for details Thank you, Radovan
regards, Martin
Any suggestations?
Thanks, Radovan
On 02/10/2014 05:25 PM, Martin Kersten wrote:
Hello
It is not a crash as such, but a safeguard against an incorrect environment. A database logger should never be removed, because it is needed to realise transaction properties.
The log message indicates what to do.
regards, Martin
On 10/02/14 16:08, Radovan Bičiště wrote:
Hello, I hope anyone can help with database crash. In .gdk_lock is following: USR=1001 PID=6751 TIME=Mon Feb 10 16:04:17 2014 @ LOGON USR=1001 PID=6751 TIME=Mon Feb 10 16:04:18 2014 @ !FATAL: logger_new: there is a logger catalog, but no log file. Are you sure you are using the correct combination of database (--dbpath) and log directory (--set sql_logdir)? If you have done a recent update of the server, it may be that your logs are in an old location. You should then either use --set sql_logdir=<path to old log directory> or move the old log directory to the new location (/u01/ceosdata/database/monetdb/drmax/crm/zkdev1/sql_logs/sql/). USR=1001 PID=6751 TIME=Mon Feb 10 16:04:18 2014 @ LOGOFF
Anyway to start the database without logs ? Thanks, Radovan
-- __________________________ Radovan Bičiště ceos data s.r.o. Pouchovská 153 500 03 Hradec Králové Czech Republic mobil CZ: +420 601 563 014 skype: rbiciste
On 11/02/14 11:39, Radovan Bičiště wrote: try to run this command gdb --args /usr/bin/mserver5 --dbpath=/u01/ceosdata/database/monetdb/drmax/crm/zkdev1 --set merovingian_uri=mapi:monetdb://lux-analytics:50000/zkdev1 --set mapi_open=false --set mapi_port=0 --set mapi_usock=/u01/ceosdata/database/monetdb/drmax/crm/zkdev1/.mapi.sock --set monet_vault_key=/u01/ceosdata/database/monetdb/drmax/crm/zkdev1/.vaultkey --set gdk_nr_threads=4 --set max_clients=64 --set sql_optimizer=default_pipe --dbinit="sql.start();" this brings up the debugger. Then type the command: run If you see a SEGFAULT, then type the command thr apply all where and sent the stacktrace . Thanks, Martin
Thank you Martin for help. But no stacktrace was generated. Here is output from gdb: ceos@lux-analytics:~$ gdb --args /usr/bin/mserver5 --dbpath=/u01/ceosdata/database/monetdb/drmax/crm/zkdev1 --set merovingian_uri=mapi:monetdb://lux-analytics:50000/zkdev1 --set mapi_open=false --set mapi_port=0 --set mapi_usock=/u01/ceosdata/database/monetdb/drmax/crm/zkdev1/.mapi.sock --set monet_vault_key=/u01/ceosdata/database/monetdb/drmax/crm/zkdev1/.vaultkey --set gdk_nr_threads=4 --set max_clients=64 --set sql_optimizer=default_pipe --dbinit="sql.start();" GNU gdb (Ubuntu/Linaro 7.4-2012.04-0ubuntu2.1) 7.4-2012.04 Copyright (C) 2012 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". For bug reporting instructions, please see: http://bugs.launchpad.net/gdb-linaro/... Reading symbols from /usr/bin/mserver5...(no debugging symbols found)...done. (gdb) run Starting program: /usr/bin/mserver5 --dbpath=/u01/ceosdata/database/monetdb/drmax/crm/zkdev1 --set merovingian_uri=mapi:monetdb://lux-analytics:50000/zkdev1 --set mapi_open=false --set mapi_port=0 --set mapi_usock=/u01/ceosdata/database/monetdb/drmax/crm/zkdev1/.mapi.sock --set monet_vault_key=/u01/ceosdata/database/monetdb/drmax/crm/zkdev1/.vaultkey --set gdk_nr_threads=4 --set max_clients=64 --set sql_optimizer=default_pipe --dbinit=sql.start\(\)\; [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". [New Thread 0x7fffeed87700 (LWP 1996)] # MonetDB 5 server v11.15.11 "Feb2013-SP3" # Serving database 'zkdev1', using 4 threads # Compiled for x86_64-pc-linux-gnu/64bit with 64bit OIDs dynamically linked # Found 47.174 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2013 MonetDB B.V., all rights reserved # Visit http://www.monetdb.org/ for further information [New Thread 0x7fffeeb85700 (LWP 1997)] [New Thread 0x7fffedeeb700 (LWP 1998)] # Listening for UNIX domain connection requests on mapi:monetdb:///u01/ceosdata/database/monetdb/drmax/crm/zkdev1/.mapi.sock Program received signal SIGSEGV, Segmentation fault. 0x00007fffee5ee02b in delta_bind_bat () from /usr/lib/monetdb5/lib_sql.so (gdb) thr sent the stacktrace No symbol table is loaded. Use the "file" command. (gdb) Radovan On 02/11/2014 11:42 AM, Martin Kersten wrote:
On 11/02/14 11:39, Radovan Bičiště wrote:
try to run this command gdb --args /usr/bin/mserver5 --dbpath=/u01/ceosdata/database/monetdb/drmax/crm/zkdev1 --set merovingian_uri=mapi:monetdb://lux-analytics:50000/zkdev1 --set mapi_open=false --set mapi_port=0 --set mapi_usock=/u01/ceosdata/database/monetdb/drmax/crm/zkdev1/.mapi.sock --set monet_vault_key=/u01/ceosdata/database/monetdb/drmax/crm/zkdev1/.vaultkey --set gdk_nr_threads=4 --set max_clients=64 --set sql_optimizer=default_pipe --dbinit="sql.start();"
this brings up the debugger. Then type the command:
run
If you see a SEGFAULT, then type the command thr apply all where
and sent the stacktrace . Thanks, Martin
_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
-- __________________________ Radovan Bičiště ceos data s.r.o. Pouchovská 153 500 03 Hradec Králové Czech Republic mobil CZ: +420 601 563 014 skype: rbiciste
participants (3)
-
Martin Kersten -
Martin Kersten -
Radovan Bičiště