log file not found after a crash

Medha Atre

13 Oct 2012 13 Oct '12

2:26 p.m.

Hi, I am running MonetDB v. 11.9.7. I have a large RDF dataset loaded in it (800+ million triples) as separate predicate tables for each unique predicate. I am running some simple one join SPAQRL queries ("?s :p1 ?o JOIN ?o :p2 ?x" kind) in bulk through a shell script (echo "<some query>" | mclient -drdf -lsql >> results). Sometime overnight when the queries were running, an error occurred in the database run, and it crashed by spitting out messages "monetdbd: an internal error has occurred, refer to the logs for details, please try again later" and "monetdbd: internal error while starting mserver, please refer to the logs" for the rest of the queries after some query caused this error. But when I tried to open "<Path To MonetDB installation>/var/log/monetdb/merovingian.log", it said "no such file or directory". I restarted the server and the particular database. It shows me its status as "health 97%" and "crash <crash date and time>". I tried to find the logfile again, but it's not there. I am wondering how I can know what the problem was. _______________________________________________ users-list mailing list users-list@monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

Show replies by date

Martin Kersten

13 Oct 13 Oct

2:43 p.m.

Hello select * from environment() gives an overview of the locations of things. The merovingian.log should recide in the gdk_dbfarm folder If your disk/system experienced a hard crash, then both database and log may be broken. Then you have to recover from backup regars, Martin On 10/13/12 4:26 PM, Medha Atre wrote:

...

Hi,

I am running MonetDB v. 11.9.7. I have a large RDF dataset loaded in it (800+ million triples) as separate predicate tables for each unique predicate. I am running some simple one join SPAQRL queries ("?s :p1 ?o JOIN ?o :p2 ?x" kind) in bulk through a shell script (echo "<some query>" | mclient -drdf -lsql >> results). Sometime overnight when the queries were running, an error occurred in the database run, and it crashed by spitting out messages

"monetdbd: an internal error has occurred, refer to the logs for details, please try again later" and "monetdbd: internal error while starting mserver, please refer to the logs"

for the rest of the queries after some query caused this error.

But when I tried to open "<Path To MonetDB installation>/var/log/monetdb/merovingian.log", it said "no such file or directory". I restarted the server and the particular database. It shows me its status as "health 97%" and "crash <crash date and time>". I tried to find the logfile again, but it's not there.

I am wondering how I can know what the problem was. _______________________________________________ users-list mailing list users-list@monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

_______________________________________________ users-list mailing list users-list@monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

Medha Atre

20 Oct 20 Oct

11:31 a.m.

This time it again crashed while running queries in bulk. I had written a script to restart the server after every 50 queries because the last time I had also got a message "too many open files" on the command prompt after the MonetDB crash. This time merovingian.log under under my dbfarm showed messages like "database 'uniprot' (3469) was killed by signal SIGBUS". I also checked /var/syslog and saw messages like -- "INFO: task mserver5:4006 blocked for more than 120 seconds.", ""echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message." with timestamp around the time MonetDB had crashed, with a "Call trace:" given after that. On Sat, Oct 13, 2012 at 10:43 AM, Martin Kersten wrote:

...

Hello

select * from environment()

gives an overview of the locations of things. The merovingian.log should recide in the gdk_dbfarm folder

If your disk/system experienced a hard crash, then both database and log may be broken. Then you have to recover from backup

regars, Martin

On 10/13/12 4:26 PM, Medha Atre wrote:

...
Hi,

I am running MonetDB v. 11.9.7. I have a large RDF dataset loaded in it (800+ million triples) as separate predicate tables for each unique predicate. I am running some simple one join SPAQRL queries ("?s :p1 ?o JOIN ?o :p2 ?x" kind) in bulk through a shell script (echo "<some query>" | mclient -drdf -lsql >> results). Sometime overnight when the queries were running, an error occurred in the database run, and it crashed by spitting out messages

"monetdbd: an internal error has occurred, refer to the logs for details, please try again later" and "monetdbd: internal error while starting mserver, please refer to the logs"

for the rest of the queries after some query caused this error.

But when I tried to open "<Path To MonetDB installation>/var/log/monetdb/merovingian.log", it said "no such file or directory". I restarted the server and the particular database. It shows me its status as "health 97%" and "crash <crash date and time>". I tried to find the logfile again, but it's not there.

I am wondering how I can know what the problem was. _______________________________________________ users-list mailing list users-list@monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

_______________________________________________ users-list mailing list users-list@monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

users-list mailing list users-list@monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

Medha Atre

22 Oct 22 Oct

3:55 a.m.

Just to give some more information: MonetDB 11.9.7 repeatedly crashes on some long running queries. There is no problem with the HDD, because the rest of the programs and OS work fine. For the long running queries, I see the physical memory consumption going on increasingly steadily and then MonetDB crashes by printing message on the command prompt -- MAPI = (monetdb) /tmp/.s.monetdb.5678 ACTION= read_line QUERY = select count(t1.sub||t1.obj||t2.sub) from uniprot800m_p37_obj as t1, uniprot800m_p70_obj as t2 where t1.obj=t2.obj; ERROR = !Connection terminated Tables uniprot800m_p37_obj and uniprot800m_p70_obj are created by ordering on column "obj" first and then on "sub" (so the query is actually a simple merge-join). Also another thing noted was -- when MonetDB starts crashing, it ends up consuming almost entire remaining empty space on the disk (the original datasize is 48GB, but it consumes additional 150+ GB filling up the entire disk). I am wondering if there is something wrong with the way query is written or if there is some other problem.. On Sat, Oct 20, 2012 at 7:31 AM, Medha Atre wrote:

...

This time it again crashed while running queries in bulk. I had written a script to restart the server after every 50 queries because the last time I had also got a message "too many open files" on the command prompt after the MonetDB crash.

This time merovingian.log under under my dbfarm showed messages like "database 'uniprot' (3469) was killed by signal SIGBUS".

I also checked /var/syslog and saw messages like -- "INFO: task mserver5:4006 blocked for more than 120 seconds.", ""echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message." with timestamp around the time MonetDB had crashed, with a "Call trace:" given after that.

On Sat, Oct 13, 2012 at 10:43 AM, Martin Kersten wrote:

...
Hello

select * from environment()

gives an overview of the locations of things. The merovingian.log should recide in the gdk_dbfarm folder

If your disk/system experienced a hard crash, then both database and log may be broken. Then you have to recover from backup

regars, Martin

On 10/13/12 4:26 PM, Medha Atre wrote:

...
Hi,

I am running MonetDB v. 11.9.7. I have a large RDF dataset loaded in it (800+ million triples) as separate predicate tables for each unique predicate. I am running some simple one join SPAQRL queries ("?s :p1 ?o JOIN ?o :p2 ?x" kind) in bulk through a shell script (echo "<some query>" | mclient -drdf -lsql >> results). Sometime overnight when the queries were running, an error occurred in the database run, and it crashed by spitting out messages

"monetdbd: an internal error has occurred, refer to the logs for details, please try again later" and "monetdbd: internal error while starting mserver, please refer to the logs"

for the rest of the queries after some query caused this error.

But when I tried to open "<Path To MonetDB installation>/var/log/monetdb/merovingian.log", it said "no such file or directory". I restarted the server and the particular database. It shows me its status as "health 97%" and "crash <crash date and time>". I tried to find the logfile again, but it's not there.

I am wondering how I can know what the problem was. _______________________________________________ users-list mailing list users-list@monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

_______________________________________________ users-list mailing list users-list@monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

users-list mailing list users-list@monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

Fabian Groffen

4:51 p.m.

On 21-10-2012 23:55:12 -0400, Medha Atre wrote:

...

Just to give some more information: MonetDB 11.9.7 repeatedly crashes on some long running queries. There is no problem with the HDD, because the rest of the programs and OS work fine.

I'd be very interested to know how this is working on 11.13.3 (a.k.a. Oct2012). Fabian _______________________________________________ users-list mailing list users-list@monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

Medha Atre

4:54 p.m.

Is it out by now? I don't see 11.13.3 available for download on the MonetDB website. Latest I am seeing is 11.11.11 (Sept'12). On Mon, Oct 22, 2012 at 12:51 PM, Fabian Groffen wrote:

...

On 21-10-2012 23:55:12 -0400, Medha Atre wrote:

...
Just to give some more information: MonetDB 11.9.7 repeatedly crashes on some long running queries. There is no problem with the HDD, because the rest of the programs and OS work fine.

I'd be very interested to know how this is working on 11.13.3 (a.k.a. Oct2012).

Fabian

users-list mailing list users-list@monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

Medha Atre

4:55 p.m.

Ok I found it under 'testing'. http://dev.monetdb.org/downloads/testing/sources/Oct2012/ Is it considered stable right now? If not, when is the earliest it can be available? On Mon, Oct 22, 2012 at 12:54 PM, Medha Atre wrote:

...

Is it out by now? I don't see 11.13.3 available for download on the MonetDB website. Latest I am seeing is 11.11.11 (Sept'12).

On Mon, Oct 22, 2012 at 12:51 PM, Fabian Groffen wrote:

...
On 21-10-2012 23:55:12 -0400, Medha Atre wrote:

...
Just to give some more information: MonetDB 11.9.7 repeatedly crashes on some long running queries. There is no problem with the HDD, because the rest of the programs and OS work fine.

I'd be very interested to know how this is working on 11.13.3 (a.k.a. Oct2012).

Fabian

users-list mailing list users-list@monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

Fabian Groffen

5:12 p.m.

On 22-10-2012 12:55:56 -0400, Medha Atre wrote:

...

Ok I found it under 'testing'. http://dev.monetdb.org/downloads/testing/sources/Oct2012/ Is it considered stable right now? If not, when is the earliest it can be available?

Tomorrow by my planning. _______________________________________________ users-list mailing list users-list@monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

Ashish Kumar Singh

5:48 p.m.

HI Fabian and Medha, What we have seen from previous released version of monetdb is that it's not usually happy with subquerie and increases memory usage? Regards, Ashish -----Original Message----- From: users-list-bounces+ashishk=altair.com@monetdb.org [mailto:users-list-bounces+ashishk=altair.com@monetdb.org] On Behalf Of Fabian Groffen Sent: Monday, October 22, 2012 10:43 PM To: Communication channel for MonetDB users Subject: Re: log file not found after a crash On 22-10-2012 12:55:56 -0400, Medha Atre wrote:

...

Ok I found it under 'testing'. http://dev.monetdb.org/downloads/testing/sources/Oct2012/ Is it considered stable right now? If not, when is the earliest it can be available?

Tomorrow by my planning. _______________________________________________ users-list mailing list users-list@monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

Medha Atre

23 Oct 23 Oct

1:26 p.m.

I see that --enable-bits is now obsolete in this release. So are the users required to set "CC" env variable or "CFLAGS" (e.g. CLAGS="-m64", I am confused because the configure script says set CC variable)? If I am compiling on a 64bit OS with 64 bit architecture, do I need to give this option to configure script explicitly or is it by default compiled as a 64-bit code? I compared the configure script of latest release and previous 11.9.7 and 11.9.7 configure has some elaborate logic after "--enable-bit" checking. So I am tad confused. I am on Ubuntu 12.04 distro of Linux with "gcc" as the default compiler. On Mon, Oct 22, 2012 at 1:12 PM, Fabian Groffen wrote:

...

On 22-10-2012 12:55:56 -0400, Medha Atre wrote:

...
Ok I found it under 'testing'. http://dev.monetdb.org/downloads/testing/sources/Oct2012/ Is it considered stable right now? If not, when is the earliest it can be available?

Tomorrow by my planning.

users-list mailing list users-list@monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

Stefan Manegold

1:37 p.m.

The default is whatever your system's / environment's default compiler (-configuration) is. --enable-bits has been abondoned, because it did not work reliably in all case, as there are compilers that we are not aware of or that need other measure than a commandline option to change the ABI/code they generate, e.g., need to be called by different name, etc. Thus, if default configure (without setting CC) gives you want you want, then you're just fine and don't need to do/set anything else. If you need something different than what you get as default (i.e., generate 32-bit binaries rather than 64-bit, or vice versa), then you need to tell configure with other compiler (-options) to use, e.g., by setting your PATH or CC variable accordingly. Best, Stefan -- | Stefan.Manegold @ CWI.nl | DB Architectures (INS1) | | http://CWI.nl/~manegold/ | Science Park 123 (L321) | | Tel.: +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) | ----- Original Message -----

...

I see that --enable-bits is now obsolete in this release. So are the users required to set "CC" env variable or "CFLAGS" (e.g. CLAGS="-m64", I am confused because the configure script says set CC variable)? If I am compiling on a 64bit OS with 64 bit architecture, do I need to give this option to configure script explicitly or is it by default compiled as a 64-bit code? I compared the configure script of latest release and previous 11.9.7 and 11.9.7 configure has some elaborate logic after "--enable-bit" checking.

So I am tad confused. I am on Ubuntu 12.04 distro of Linux with "gcc" as the default compiler.

On Mon, Oct 22, 2012 at 1:12 PM, Fabian Groffen wrote:

...
On 22-10-2012 12:55:56 -0400, Medha Atre wrote:

...
Ok I found it under 'testing'. http://dev.monetdb.org/downloads/testing/sources/Oct2012/ Is it considered stable right now? If not, when is the earliest it can be available?

Tomorrow by my planning.

users-list mailing list users-list@monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

_______________________________________________ users-list mailing list users-list@monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

4448

Age (days ago)

4458

Last active (days ago)

List overview

Download

10 comments

5 participants

participants (5)

Ashish Kumar Singh
Fabian Groffen
Martin Kersten
Medha Atre
Stefan Manegold

log file not found after a crash

tags

participants (5)