Hey folks, had a database running for several weeks without any problems but early this morning anytime a client would try to connect to it the connection would just hang. After shutting down and restarting I see this:
2009-11-23 09:38:06 MSG merovingian[28036]: starting Merovingian 1.1 for dbfarm
/opt/MonetDB5/dbfarm
2009-11-23 09:38:06 MSG merovingian[28036]: listening for TCP connections on
0.0.0.0:50000
2009-11-23 09:38:06 MSG discovery[28036]: listening for UDP messages on
0.0.0.0:50000
2009-11-23 09:38:06 MSG merovingian[28036]: handling commands over UNIX socket
/opt/MonetDB5/dbfarm/.merovingian_control
2009-11-23 09:38:06 MSG discovery[28036]: new neighbour 10.1.0.79
2009-11-23 09:38:07 MSG discovery[28036]: new database mapi:monetdb://
hsdw01:50000/reporting_primary (ttl=660s)
2009-11-23 09:39:45 MSG merovingian[28036]: starting database
'reporting_primary' due to control signal
2009-11-23 09:39:45 MSG merovingian[28036]: database 'reporting_primary' has
crashed after start on 2009-11-23 09:36:59, attempting restart, up min/avg/max:
2m/2d/1w, crash average: 1.00 0.60 0.27 (18-10=8)
2009-11-23 09:39:46 MSG reporting_primary[28067]: arguments: /usr/bin/mserver5
--config=/etc/monetdb5.conf --dbname=reporting_primary --dbinit=include sql;
--set monet_daemon=yes --set mapi_open=false --set mapi_autosense=true --set
mapi_port=50001 --set monet_vault_key=/opt/MonetDB5/dbfarm/reporting_primary
/.vaultkey
2009-11-23 09:39:46 MSG reporting_primary[28067]: # MonetDB server v5.14.2,
based on kernel v1.32.2
2009-11-23 09:39:46 MSG reporting_primary[28067]: # Serving database
'reporting_primary', using 8 threads
2009-11-23 09:39:46 MSG reporting_primary[28067]: # Compiled for
x86_64-pc-linux-gnu/64bit with 64bit OIDs dynamically linked
2009-11-23 09:39:46 MSG reporting_primary[28067]: # Copyright (c) 1993-July
2008 CWI.
2009-11-23 09:39:46 MSG reporting_primary[28067]: # Copyright (c) August
2008-2009 MonetDB B.V., all rights reserved
2009-11-23 09:39:46 MSG reporting_primary[28067]: # Visit http://monetdb.cwi.nl
/ for further information
2009-11-23 09:39:46 MSG reporting_primary[28067]: # Listening for connection
requests on mapi:monetdb://127.0.0.1:50001/
2009-11-23 09:39:46 MSG reporting_primary[28067]: # MonetDB/SQL module v2.32.2
loaded
2009-11-23 09:39:46 ERR reporting_primary[28067]: !mvc_init: unable to create
system tables
2009-11-23 09:39:55 ERR merovingian[28036]: failed to fork mserver: database
'reporting_primary' has inconsistent state (running but dead), review
merovingian's logfile for any peculiarities
The database marks itself as running but any attempt to connect to it returns the following error:
monetdb ERROR MALException:setScenario:Scenario not initialized 'sql'
I've read other problems reports that had a similar issue and the suggestion was to wait for the sql module to allow connections but even after 5 minutes this is all that is returned. Is there any way to get this database back to a consistent state so that it is usable?
73,
Matthew W. Jones (KI4ZIB)
http://matburt.net