[MonetDB-users] copy into file
Hi, It looks like the same problem exists no matter what the OS (Fedora 12 or Ubuntu 9.10 or Windows) and this worked in Nov2009-SP1 release. When you try: sql>copy select * from table1 limit 1000 into '/tmp/output.txt'; MAPI = monetdb@localhost:50000 ACTION= read_line QUERY = copy select * from table1 limit 1000 into '/tmp/output.txt'; ERROR = !Connection terminated Timer 1105.876 msec You have mail in /var/spool/mail/root [root@monetdb ~]# [root@monetdb /]# mserver5 --version MonetDB server v5.18.3 (64-bit), based on kernel v1.36.3 (64-bit oids) Copyright (c) 1993-July 2008 CWI Copyright (c) August 2008-2010 MonetDB B.V., all rights reserved Visit http://monetdb.cwi.nl/ for further information Found 47.3GiB available memory, 16 available cpu cores Configured for prefix: /usr Libraries: libpcre: 7.8 2008-09-05 (compiled with 7.8) openssl: OpenSSL 1.0.0-beta4 10 Nov 2009 (compiled with OpenSSL 1.0.0-fips-beta4 10 Nov 2009) libxml2: 2.7.6 (compiled with 2.7.6) Compiled by: mockbuild@ (x86_64-redhat-linux-gnu) Compilation: gcc -O2 -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -O6 -fomit-frame-pointer -finline-functions -falign-loops=4 -falign-jumps=4 -falign-functions=4 -fexpensive-optimizations -funroll-loops -frerun-cse-after-loop -frerun-loop-opt -ftree-vectorize Linking : ld -IPA -m elf_x86_64 Please help. Thanks. Dariusz.
On 21-04-2010 13:07:37 -0400, dariuszs wrote:
Hi, It looks like the same problem exists no matter what the OS (Fedora 12 or Ubuntu 9.10 or Windows) and this worked in Nov2009-SP1 release. When you try: sql>copy select * from table1 limit 1000 into '/tmp/output.txt'; MAPI = monetdb@localhost:50000 ACTION= read_line QUERY = copy select * from table1 limit 1000 into '/tmp/output.txt'; ERROR = !Connection terminated Timer 1105.876 msec
please check your merovingian.log or provide us with the console output of the mserver5 process.
[root@monetdb /]# mserver5 --version MonetDB server v5.18.3 (64-bit), based on kernel v1.36.3 (64-bit oids) Copyright (c) 1993-July 2008 CWI Copyright (c) August 2008-2010 MonetDB B.V., all rights reserved Visit http://monetdb.cwi.nl/ for further information Found 47.3GiB available memory, 16 available cpu cores Configured for prefix: /usr Libraries: libpcre: 7.8 2008-09-05 (compiled with 7.8) openssl: OpenSSL 1.0.0-beta4 10 Nov 2009 (compiled with OpenSSL 1.0.0-fips-beta4 10 Nov 2009) libxml2: 2.7.6 (compiled with 2.7.6) Compiled by: mockbuild@ (x86_64-redhat-linux-gnu) Compilation: gcc -O2 -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -O6 -fomit-frame-pointer -finline-functions -falign-loops=4 -falign-jumps=4 -falign-functions=4 -fexpensive-optimizations -funroll-loops -frerun-cse-after-loop -frerun-loop-opt -ftree-vectorize Linking : ld -IPA -m elf_x86_64
[root@monetdb /]# mserver5 --dbinit="include sql;" # MonetDB server v5.18.3, based on kernel v1.36.3 # Serving database 'demo', using 16 threads # Compiled for x86_64-redhat-linux-gnu/64bit with 64bit OIDs dynamically linked # Found 47.259 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2010 MonetDB B.V., all rights reserved # Visit http://monetdb.cwi.nl/ for further information #warning: please don't forget to set your vault key! #(see /etc/monetdb5.conf) # Listening for connection requests on mapi:monetdb://127.0.0.1:50000/ # MonetDB/SQL module v2.36.3 loaded
Segmentation fault [root@monetdb /]#
Is this what you had in mind? Thanks. Dariusz. Fabian Groffen wrote:
On 21-04-2010 13:07:37 -0400, dariuszs wrote:
Hi, It looks like the same problem exists no matter what the OS (Fedora 12 or Ubuntu 9.10 or Windows) and this worked in Nov2009-SP1 release. When you try: sql>copy select * from table1 limit 1000 into '/tmp/output.txt'; MAPI = monetdb@localhost:50000 ACTION= read_line QUERY = copy select * from table1 limit 1000 into '/tmp/output.txt'; ERROR = !Connection terminated Timer 1105.876 msec
please check your merovingian.log or provide us with the console output of the mserver5 process.
[root@monetdb /]# mserver5 --version MonetDB server v5.18.3 (64-bit), based on kernel v1.36.3 (64-bit oids) Copyright (c) 1993-July 2008 CWI Copyright (c) August 2008-2010 MonetDB B.V., all rights reserved Visit http://monetdb.cwi.nl/ for further information Found 47.3GiB available memory, 16 available cpu cores Configured for prefix: /usr Libraries: libpcre: 7.8 2008-09-05 (compiled with 7.8) openssl: OpenSSL 1.0.0-beta4 10 Nov 2009 (compiled with OpenSSL 1.0.0-fips-beta4 10 Nov 2009) libxml2: 2.7.6 (compiled with 2.7.6) Compiled by: mockbuild@ (x86_64-redhat-linux-gnu) Compilation: gcc -O2 -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -O6 -fomit-frame-pointer -finline-functions -falign-loops=4 -falign-jumps=4 -falign-functions=4 -fexpensive-optimizations -funroll-loops -frerun-cse-after-loop -frerun-loop-opt -ftree-vectorize Linking : ld -IPA -m elf_x86_64
------------------------------------------------------------------------------ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
On 21-04-2010 13:20:17 -0400, dariuszs wrote:
[root@monetdb /]# mserver5 --dbinit="include sql;" # MonetDB server v5.18.3, based on kernel v1.36.3 # Serving database 'demo', using 16 threads # Compiled for x86_64-redhat-linux-gnu/64bit with 64bit OIDs dynamically linked # Found 47.259 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2010 MonetDB B.V., all rights reserved # Visit http://monetdb.cwi.nl/ for further information #warning: please don't forget to set your vault key! #(see /etc/monetdb5.conf) # Listening for connection requests on mapi:monetdb://127.0.0.1:50000/ # MonetDB/SQL module v2.36.3 loaded
Segmentation fault [root@monetdb /]#
Is this what you had in mind? Thanks. Dariusz.
Yeah, it just segfaults ;)
Hi, You sound like it's good news, but where do we go from here? Thanks. Dariusz. Fabian Groffen wrote:
On 21-04-2010 13:20:17 -0400, dariuszs wrote:
[root@monetdb /]# mserver5 --dbinit="include sql;" # MonetDB server v5.18.3, based on kernel v1.36.3 # Serving database 'demo', using 16 threads # Compiled for x86_64-redhat-linux-gnu/64bit with 64bit OIDs dynamically linked # Found 47.259 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2010 MonetDB B.V., all rights reserved # Visit http://monetdb.cwi.nl/ for further information #warning: please don't forget to set your vault key! #(see /etc/monetdb5.conf) # Listening for connection requests on mapi:monetdb://127.0.0.1:50000/ # MonetDB/SQL module v2.36.3 loaded
Segmentation fault [root@monetdb /]#
Is this what you had in mind? Thanks. Dariusz.
Yeah, it just segfaults ;)
------------------------------------------------------------------------------ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
On 21-04-2010 13:32:42 -0400, dariuszs wrote:
Hi, You sound like it's good news, but where do we go from here? Thanks.
Sorry to give you the impression it was good news. It was bad news, since your server seems to be crashing without giving any additional information to give us a clue why.
Hi, I'm using Fedora 12 on this machine, are you saying you don't have this problem? I've replicated this problem also on Ubuntu 9.10 and Windows 2003. Thanks. Dariusz. Fabian Groffen wrote:
On 21-04-2010 13:32:42 -0400, dariuszs wrote:
Hi, You sound like it's good news, but where do we go from here? Thanks.
Sorry to give you the impression it was good news.
It was bad news, since your server seems to be crashing without giving any additional information to give us a clue why.
------------------------------------------------------------------------------ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
On 21-04-2010 13:51:52 -0400, dariuszs wrote:
Hi, I'm using Fedora 12 on this machine, are you saying you don't have this problem? I've replicated this problem also on Ubuntu 9.10 and Windows 2003. Thanks. Dariusz.
Can you try the following to see if it makes any difference? monetdb destroy demo monetdb create demo monetdb set nrthreads=1 demo monetdb release demo [load data] Alternatively try this (but I think it has very little chance of making any difference): monetdb destroy demo monetdb create demo monetdb set optpipe=inline,remap,deadcode,multiplex demo monetdb release demo [load data]
Hi, Single thread works, but what does it really mean? Thanks. Dariusz Fabian Groffen wrote:
On 21-04-2010 13:51:52 -0400, dariuszs wrote:
Hi, I'm using Fedora 12 on this machine, are you saying you don't have this problem? I've replicated this problem also on Ubuntu 9.10 and Windows 2003. Thanks. Dariusz.
Can you try the following to see if it makes any difference?
monetdb destroy demo monetdb create demo monetdb set nrthreads=1 demo monetdb release demo [load data]
Alternatively try this (but I think it has very little chance of making any difference):
monetdb destroy demo monetdb create demo monetdb set optpipe=inline,remap,deadcode,multiplex demo monetdb release demo [load data]
------------------------------------------------------------------------------ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
On 21-04-2010 14:15:33 -0400, dariuszs wrote:
Single thread works, but what does it really mean? Thanks. Dariusz
It means you likely have the same problem as Hering Cheng, which seems related to parallelism. You can try to unset (monetdb inherit nrthreads demo) the single threadedness once you have loaded the data.
On Wed, Apr 21, 2010 at 08:23:06PM +0200, Fabian Groffen wrote:
On 21-04-2010 14:15:33 -0400, dariuszs wrote:
Single thread works, but what does it really mean? Thanks. Dariusz
It means you likely have the same problem as Hering Cheng, which seems related to parallelism.
You can try to unset (monetdb inherit nrthreads demo) the single threadedness once you have loaded the data.
Just to make sure, Dariusz' problem is not with loading data from .csv file into the database (copy into from file) but with dumping data from the database into a .csv file (copy select ... into file). Hence, is it not necessarily related to Hering Cheng's loading problem. I just tried a (simplified) dump like copy select * from tables limit 10 into '/tmp/output.txt'; on my 4-core Febdora 12 desktop with Feb2010, Feb2010-SP1 & Feb2010-SP2 preview, and all work fine. Dariusz, could you please test & report whether this simple version also works (in standard multi-threaded mode) also on your 16-core machine(s)? Could you also check & report, wether the select part (without copy ... into file) works fine on your machine(s), i.e., select * from table1 limit 1000; Thanks! Stefan
------------------------------------------------------------------------------ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
-- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4199 |
Hi, I'm working on it, please wait.. Thanks. Dariusz. Stefan Manegold wrote:
On Wed, Apr 21, 2010 at 08:23:06PM +0200, Fabian Groffen wrote:
On 21-04-2010 14:15:33 -0400, dariuszs wrote:
Single thread works, but what does it really mean? Thanks. Dariusz
It means you likely have the same problem as Hering Cheng, which seems related to parallelism.
You can try to unset (monetdb inherit nrthreads demo) the single threadedness once you have loaded the data.
Just to make sure, Dariusz' problem is not with loading data from .csv file into the database (copy into from file) but with dumping data from the database into a .csv file (copy select ... into file). Hence, is it not necessarily related to Hering Cheng's loading problem.
I just tried a (simplified) dump like
copy select * from tables limit 10 into '/tmp/output.txt';
on my 4-core Febdora 12 desktop with Feb2010, Feb2010-SP1 & Feb2010-SP2 preview, and all work fine.
Dariusz,
could you please test & report whether this simple version also works (in standard multi-threaded mode) also on your 16-core machine(s)?
Could you also check & report, wether the select part (without copy ... into file) works fine on your machine(s), i.e., select * from table1 limit 1000;
Thanks!
Stefan
------------------------------------------------------------------------------ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
Hi, OK, simple query works and there is something new - when I import data and run the query in the same session - it works, when I restart the server It segfaults - file permissions? Take a look. [root@monetdb masters]# mclient -lsql -d demo user(root):monetdb password: Welcome to mclient, the MonetDB/SQL interactive terminal Database: MonetDB v5.18.3, 'demo' Type \q to quit, \? for a list of available commands auto commit mode: on sql>\< /vol2/monetdb_data/masters/create_plk100.sql operation successful sql>copy 1000000 records into plk100 from '/vol2/monetdb_data/masters/PLK100.DAT' delimiters '|','\n',''; 1000000 affected rows sql>copy select * from tables limit 10 into '/tmp/output.txt'; 10 affected rows sql>copy select * from tables limit 10 into '/tmp/output.txt'; 10 affected rows sql>copy select * from plk100 limit 10 into '/tmp/output.txt'; 10 affected rows sql>copy select * from plk100 limit 10 into '/tmp/output.txt'; 10 affected rows sql>\q [root@monetdb MonetDB]# mserver5 --dbinit="include sql;" # MonetDB server v5.18.3, based on kernel v1.36.3 # Serving database 'demo', using 16 threads # Compiled for x86_64-redhat-linux-gnu/64bit with 64bit OIDs dynamically linked # Found 47.259 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2010 MonetDB B.V., all rights reserved # Visit http://monetdb.cwi.nl/ for further information #warning: please don't forget to set your vault key! #(see /etc/monetdb5.conf) # Listening for connection requests on mapi:monetdb://127.0.0.1:50000/ # MonetDB/SQL module v2.36.3 loaded
\q
[root@monetdb masters]# mclient -lsql -d demo user(root):monetdb password: Welcome to mclient, the MonetDB/SQL interactive terminal Database: MonetDB v5.18.3, 'demo' Type \q to quit, \? for a list of available commands auto commit mode: on sql>copy select * from tables limit 10 into '/tmp/output.txt'; 10 affected rows sql>copy select * from plk100 limit 10 into '/tmp/output.txt'; MAPI = monetdb@localhost:50000 ACTION= read_line QUERY = copy select * from plk100 limit 10 into '/tmp/output.txt'; ERROR = !Connection terminated [root@monetdb masters]# [root@monetdb MonetDB]# mserver5 --dbinit="include sql;" # MonetDB server v5.18.3, based on kernel v1.36.3 # Serving database 'demo', using 16 threads # Compiled for x86_64-redhat-linux-gnu/64bit with 64bit OIDs dynamically linked # Found 47.259 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2010 MonetDB B.V., all rights reserved # Visit http://monetdb.cwi.nl/ for further information #warning: please don't forget to set your vault key! #(see /etc/monetdb5.conf) # Listening for connection requests on mapi:monetdb://127.0.0.1:50000/ # MonetDB/SQL module v2.36.3 loaded
Segmentation fault
Thanks. Dariusz. Stefan Manegold wrote:
On Wed, Apr 21, 2010 at 08:23:06PM +0200, Fabian Groffen wrote:
On 21-04-2010 14:15:33 -0400, dariuszs wrote:
Single thread works, but what does it really mean? Thanks. Dariusz
It means you likely have the same problem as Hering Cheng, which seems related to parallelism.
You can try to unset (monetdb inherit nrthreads demo) the single threadedness once you have loaded the data.
Just to make sure, Dariusz' problem is not with loading data from .csv file into the database (copy into from file) but with dumping data from the database into a .csv file (copy select ... into file). Hence, is it not necessarily related to Hering Cheng's loading problem.
I just tried a (simplified) dump like
copy select * from tables limit 10 into '/tmp/output.txt';
on my 4-core Febdora 12 desktop with Feb2010, Feb2010-SP1 & Feb2010-SP2 preview, and all work fine.
Dariusz,
could you please test & report whether this simple version also works (in standard multi-threaded mode) also on your 16-core machine(s)?
Could you also check & report, wether the select part (without copy ... into file) works fine on your machine(s), i.e., select * from table1 limit 1000;
Thanks!
Stefan
------------------------------------------------------------------------------ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
Hi, I just tried your example, i.e., loading via copy into from file then dumping via copy select ... limit ... into file then restarting server and dumping again --- obviously not with your data, though, but with ~400000 rows of a 90+ column table, and that works fine with Feb2010, Feb2010-SP1 & Feb2010-SP2 preview on my 64-bit 4-core Fedora 12 desktop. In case you are familiar with C development and debugging, you might want to compile a debugging version of MonetDB (configured with --enable-debug), run mserver5 in a debugger, locate the actual location of and reason for the segfault, and report. Remotely, I cannot do more since I cannot reproduce the segfault. Stefan On Thu, Apr 22, 2010 at 10:53:44AM -0400, dariuszs wrote:
Hi, OK, simple query works and there is something new - when I import data and run the query in the same session - it works, when I restart the server It segfaults - file permissions? Take a look.
[root@monetdb masters]# mclient -lsql -d demo user(root):monetdb password: Welcome to mclient, the MonetDB/SQL interactive terminal Database: MonetDB v5.18.3, 'demo' Type \q to quit, \? for a list of available commands auto commit mode: on sql>\< /vol2/monetdb_data/masters/create_plk100.sql operation successful sql>copy 1000000 records into plk100 from '/vol2/monetdb_data/masters/PLK100.DAT' delimiters '|','\n',''; 1000000 affected rows sql>copy select * from tables limit 10 into '/tmp/output.txt'; 10 affected rows sql>copy select * from tables limit 10 into '/tmp/output.txt'; 10 affected rows sql>copy select * from plk100 limit 10 into '/tmp/output.txt'; 10 affected rows sql>copy select * from plk100 limit 10 into '/tmp/output.txt'; 10 affected rows sql>\q
[root@monetdb MonetDB]# mserver5 --dbinit="include sql;" # MonetDB server v5.18.3, based on kernel v1.36.3 # Serving database 'demo', using 16 threads # Compiled for x86_64-redhat-linux-gnu/64bit with 64bit OIDs dynamically linked # Found 47.259 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2010 MonetDB B.V., all rights reserved # Visit http://monetdb.cwi.nl/ for further information #warning: please don't forget to set your vault key! #(see /etc/monetdb5.conf) # Listening for connection requests on mapi:monetdb://127.0.0.1:50000/ # MonetDB/SQL module v2.36.3 loaded
\q
[root@monetdb masters]# mclient -lsql -d demo user(root):monetdb password: Welcome to mclient, the MonetDB/SQL interactive terminal Database: MonetDB v5.18.3, 'demo' Type \q to quit, \? for a list of available commands auto commit mode: on sql>copy select * from tables limit 10 into '/tmp/output.txt'; 10 affected rows sql>copy select * from plk100 limit 10 into '/tmp/output.txt'; MAPI = monetdb@localhost:50000 ACTION= read_line QUERY = copy select * from plk100 limit 10 into '/tmp/output.txt'; ERROR = !Connection terminated [root@monetdb masters]#
[root@monetdb MonetDB]# mserver5 --dbinit="include sql;" # MonetDB server v5.18.3, based on kernel v1.36.3 # Serving database 'demo', using 16 threads # Compiled for x86_64-redhat-linux-gnu/64bit with 64bit OIDs dynamically linked # Found 47.259 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2010 MonetDB B.V., all rights reserved # Visit http://monetdb.cwi.nl/ for further information #warning: please don't forget to set your vault key! #(see /etc/monetdb5.conf) # Listening for connection requests on mapi:monetdb://127.0.0.1:50000/ # MonetDB/SQL module v2.36.3 loaded
Segmentation fault
Thanks. Dariusz.
Stefan Manegold wrote:
On Wed, Apr 21, 2010 at 08:23:06PM +0200, Fabian Groffen wrote:
On 21-04-2010 14:15:33 -0400, dariuszs wrote:
Single thread works, but what does it really mean? Thanks. Dariusz
It means you likely have the same problem as Hering Cheng, which seems related to parallelism.
You can try to unset (monetdb inherit nrthreads demo) the single threadedness once you have loaded the data.
Just to make sure, Dariusz' problem is not with loading data from .csv file into the database (copy into from file) but with dumping data from the database into a .csv file (copy select ... into file). Hence, is it not necessarily related to Hering Cheng's loading problem.
I just tried a (simplified) dump like
copy select * from tables limit 10 into '/tmp/output.txt';
on my 4-core Febdora 12 desktop with Feb2010, Feb2010-SP1 & Feb2010-SP2 preview, and all work fine.
Dariusz,
could you please test & report whether this simple version also works (in standard multi-threaded mode) also on your 16-core machine(s)?
Could you also check & report, wether the select part (without copy ... into file) works fine on your machine(s), i.e., select * from table1 limit 1000;
Thanks!
Stefan
------------------------------------------------------------------------------ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
------------------------------------------------------------------------------ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
-- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4199 |
Hi, This is actually very good news, that means It's something my machine. If you don't mind - what's your hardware? Thanks. Dariusz. Stefan Manegold wrote:
Hi,
I just tried your example, i.e., loading via copy into from file then dumping via copy select ... limit ... into file then restarting server and dumping again --- obviously not with your data, though, but with ~400000 rows of a 90+ column table, and that works fine with Feb2010, Feb2010-SP1 & Feb2010-SP2 preview on my 64-bit 4-core Fedora 12 desktop.
In case you are familiar with C development and debugging, you might want to compile a debugging version of MonetDB (configured with --enable-debug), run mserver5 in a debugger, locate the actual location of and reason for the segfault, and report. Remotely, I cannot do more since I cannot reproduce the segfault.
Stefan
On Thu, Apr 22, 2010 at 10:53:44AM -0400, dariuszs wrote:
Hi, OK, simple query works and there is something new - when I import data and run the query in the same session - it works, when I restart the server It segfaults - file permissions? Take a look.
[root@monetdb masters]# mclient -lsql -d demo user(root):monetdb password: Welcome to mclient, the MonetDB/SQL interactive terminal Database: MonetDB v5.18.3, 'demo' Type \q to quit, \? for a list of available commands auto commit mode: on sql>\< /vol2/monetdb_data/masters/create_plk100.sql operation successful sql>copy 1000000 records into plk100 from '/vol2/monetdb_data/masters/PLK100.DAT' delimiters '|','\n',''; 1000000 affected rows sql>copy select * from tables limit 10 into '/tmp/output.txt'; 10 affected rows sql>copy select * from tables limit 10 into '/tmp/output.txt'; 10 affected rows sql>copy select * from plk100 limit 10 into '/tmp/output.txt'; 10 affected rows sql>copy select * from plk100 limit 10 into '/tmp/output.txt'; 10 affected rows sql>\q
[root@monetdb MonetDB]# mserver5 --dbinit="include sql;" # MonetDB server v5.18.3, based on kernel v1.36.3 # Serving database 'demo', using 16 threads # Compiled for x86_64-redhat-linux-gnu/64bit with 64bit OIDs dynamically linked # Found 47.259 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2010 MonetDB B.V., all rights reserved # Visit http://monetdb.cwi.nl/ for further information #warning: please don't forget to set your vault key! #(see /etc/monetdb5.conf) # Listening for connection requests on mapi:monetdb://127.0.0.1:50000/ # MonetDB/SQL module v2.36.3 loaded
\q
[root@monetdb masters]# mclient -lsql -d demo user(root):monetdb password: Welcome to mclient, the MonetDB/SQL interactive terminal Database: MonetDB v5.18.3, 'demo' Type \q to quit, \? for a list of available commands auto commit mode: on sql>copy select * from tables limit 10 into '/tmp/output.txt'; 10 affected rows sql>copy select * from plk100 limit 10 into '/tmp/output.txt'; MAPI = monetdb@localhost:50000 ACTION= read_line QUERY = copy select * from plk100 limit 10 into '/tmp/output.txt'; ERROR = !Connection terminated [root@monetdb masters]#
[root@monetdb MonetDB]# mserver5 --dbinit="include sql;" # MonetDB server v5.18.3, based on kernel v1.36.3 # Serving database 'demo', using 16 threads # Compiled for x86_64-redhat-linux-gnu/64bit with 64bit OIDs dynamically linked # Found 47.259 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2010 MonetDB B.V., all rights reserved # Visit http://monetdb.cwi.nl/ for further information #warning: please don't forget to set your vault key! #(see /etc/monetdb5.conf) # Listening for connection requests on mapi:monetdb://127.0.0.1:50000/ # MonetDB/SQL module v2.36.3 loaded
Segmentation fault
Thanks. Dariusz.
Stefan Manegold wrote:
On Wed, Apr 21, 2010 at 08:23:06PM +0200, Fabian Groffen wrote:
On 21-04-2010 14:15:33 -0400, dariuszs wrote:
Single thread works, but what does it really mean? Thanks. Dariusz
It means you likely have the same problem as Hering Cheng, which seems related to parallelism.
You can try to unset (monetdb inherit nrthreads demo) the single threadedness once you have loaded the data.
Just to make sure, Dariusz' problem is not with loading data from .csv file into the database (copy into from file) but with dumping data from the database into a .csv file (copy select ... into file). Hence, is it not necessarily related to Hering Cheng's loading problem.
I just tried a (simplified) dump like
copy select * from tables limit 10 into '/tmp/output.txt';
on my 4-core Febdora 12 desktop with Feb2010, Feb2010-SP1 & Feb2010-SP2 preview, and all work fine.
Dariusz,
could you please test & report whether this simple version also works (in standard multi-threaded mode) also on your 16-core machine(s)?
Could you also check & report, wether the select part (without copy ... into file) works fine on your machine(s), i.e., select * from table1 limit 1000;
Thanks!
Stefan
------------------------------------------------------------------------------ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
------------------------------------------------------------------------------ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
On Thu, Apr 22, 2010 at 11:34:53AM -0400, dariuszs wrote:
Hi, This is actually very good news, that means It's something my machine.
... or in your data, or in your compiler, or in your copy of the MonetDb source, or ...
If you don't mind - what's your hardware? Thanks. Dariusz.
No that I think it matters: Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz, 8 GB RAM Stefan
Stefan Manegold wrote:
Hi,
I just tried your example, i.e., loading via copy into from file then dumping via copy select ... limit ... into file then restarting server and dumping again --- obviously not with your data, though, but with ~400000 rows of a 90+ column table, and that works fine with Feb2010, Feb2010-SP1 & Feb2010-SP2 preview on my 64-bit 4-core Fedora 12 desktop.
In case you are familiar with C development and debugging, you might want to compile a debugging version of MonetDB (configured with --enable-debug), run mserver5 in a debugger, locate the actual location of and reason for the segfault, and report. Remotely, I cannot do more since I cannot reproduce the segfault.
Stefan
On Thu, Apr 22, 2010 at 10:53:44AM -0400, dariuszs wrote:
Hi, OK, simple query works and there is something new - when I import data and run the query in the same session - it works, when I restart the server It segfaults - file permissions? Take a look.
[root@monetdb masters]# mclient -lsql -d demo user(root):monetdb password: Welcome to mclient, the MonetDB/SQL interactive terminal Database: MonetDB v5.18.3, 'demo' Type \q to quit, \? for a list of available commands auto commit mode: on sql>\< /vol2/monetdb_data/masters/create_plk100.sql operation successful sql>copy 1000000 records into plk100 from '/vol2/monetdb_data/masters/PLK100.DAT' delimiters '|','\n',''; 1000000 affected rows sql>copy select * from tables limit 10 into '/tmp/output.txt'; 10 affected rows sql>copy select * from tables limit 10 into '/tmp/output.txt'; 10 affected rows sql>copy select * from plk100 limit 10 into '/tmp/output.txt'; 10 affected rows sql>copy select * from plk100 limit 10 into '/tmp/output.txt'; 10 affected rows sql>\q
[root@monetdb MonetDB]# mserver5 --dbinit="include sql;" # MonetDB server v5.18.3, based on kernel v1.36.3 # Serving database 'demo', using 16 threads # Compiled for x86_64-redhat-linux-gnu/64bit with 64bit OIDs dynamically linked # Found 47.259 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2010 MonetDB B.V., all rights reserved # Visit http://monetdb.cwi.nl/ for further information #warning: please don't forget to set your vault key! #(see /etc/monetdb5.conf) # Listening for connection requests on mapi:monetdb://127.0.0.1:50000/ # MonetDB/SQL module v2.36.3 loaded
\q
[root@monetdb masters]# mclient -lsql -d demo user(root):monetdb password: Welcome to mclient, the MonetDB/SQL interactive terminal Database: MonetDB v5.18.3, 'demo' Type \q to quit, \? for a list of available commands auto commit mode: on sql>copy select * from tables limit 10 into '/tmp/output.txt'; 10 affected rows sql>copy select * from plk100 limit 10 into '/tmp/output.txt'; MAPI = monetdb@localhost:50000 ACTION= read_line QUERY = copy select * from plk100 limit 10 into '/tmp/output.txt'; ERROR = !Connection terminated [root@monetdb masters]#
[root@monetdb MonetDB]# mserver5 --dbinit="include sql;" # MonetDB server v5.18.3, based on kernel v1.36.3 # Serving database 'demo', using 16 threads # Compiled for x86_64-redhat-linux-gnu/64bit with 64bit OIDs dynamically linked # Found 47.259 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2010 MonetDB B.V., all rights reserved # Visit http://monetdb.cwi.nl/ for further information #warning: please don't forget to set your vault key! #(see /etc/monetdb5.conf) # Listening for connection requests on mapi:monetdb://127.0.0.1:50000/ # MonetDB/SQL module v2.36.3 loaded
Segmentation fault
Thanks. Dariusz.
Stefan Manegold wrote:
On Wed, Apr 21, 2010 at 08:23:06PM +0200, Fabian Groffen wrote:
On 21-04-2010 14:15:33 -0400, dariuszs wrote:
Single thread works, but what does it really mean? Thanks. Dariusz
It means you likely have the same problem as Hering Cheng, which seems related to parallelism.
You can try to unset (monetdb inherit nrthreads demo) the single threadedness once you have loaded the data.
Just to make sure, Dariusz' problem is not with loading data from .csv file into the database (copy into from file) but with dumping data from the database into a .csv file (copy select ... into file). Hence, is it not necessarily related to Hering Cheng's loading problem.
I just tried a (simplified) dump like
copy select * from tables limit 10 into '/tmp/output.txt';
on my 4-core Febdora 12 desktop with Feb2010, Feb2010-SP1 & Feb2010-SP2 preview, and all work fine.
Dariusz,
could you please test & report whether this simple version also works (in standard multi-threaded mode) also on your 16-core machine(s)?
Could you also check & report, wether the select part (without copy ... into file) works fine on your machine(s), i.e., select * from table1 limit 1000;
Thanks!
Stefan
------------------------------------------------------------------------------ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
------------------------------------------------------------------------------ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
------------------------------------------------------------------------------ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
-- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4199 |
Hi, I did not compile from sources, compiler should be the same (I've got Fedora 12). Data - I did not get any errors loading files in the table nor running the counts... Hardware: Intel® Xeon® X5680, 3.33Ghz, 12M Cache,Turbo, HT, 1333MHz (8 cores), 48GB RAM Stefan Manegold wrote:
On Thu, Apr 22, 2010 at 11:34:53AM -0400, dariuszs wrote:
Hi, This is actually very good news, that means It's something my machine.
... or in your data, or in your compiler, or in your copy of the MonetDb source, or ...
If you don't mind - what's your hardware? Thanks. Dariusz.
No that I think it matters: Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz, 8 GB RAM
Stefan
Stefan Manegold wrote:
Hi,
I just tried your example, i.e., loading via copy into from file then dumping via copy select ... limit ... into file then restarting server and dumping again --- obviously not with your data, though, but with ~400000 rows of a 90+ column table, and that works fine with Feb2010, Feb2010-SP1 & Feb2010-SP2 preview on my 64-bit 4-core Fedora 12 desktop.
In case you are familiar with C development and debugging, you might want to compile a debugging version of MonetDB (configured with --enable-debug), run mserver5 in a debugger, locate the actual location of and reason for the segfault, and report. Remotely, I cannot do more since I cannot reproduce the segfault.
Stefan
On Thu, Apr 22, 2010 at 10:53:44AM -0400, dariuszs wrote:
Hi, OK, simple query works and there is something new - when I import data and run the query in the same session - it works, when I restart the server It segfaults - file permissions? Take a look.
[root@monetdb masters]# mclient -lsql -d demo user(root):monetdb password: Welcome to mclient, the MonetDB/SQL interactive terminal Database: MonetDB v5.18.3, 'demo' Type \q to quit, \? for a list of available commands auto commit mode: on sql>\< /vol2/monetdb_data/masters/create_plk100.sql operation successful sql>copy 1000000 records into plk100 from '/vol2/monetdb_data/masters/PLK100.DAT' delimiters '|','\n',''; 1000000 affected rows sql>copy select * from tables limit 10 into '/tmp/output.txt'; 10 affected rows sql>copy select * from tables limit 10 into '/tmp/output.txt'; 10 affected rows sql>copy select * from plk100 limit 10 into '/tmp/output.txt'; 10 affected rows sql>copy select * from plk100 limit 10 into '/tmp/output.txt'; 10 affected rows sql>\q
[root@monetdb MonetDB]# mserver5 --dbinit="include sql;" # MonetDB server v5.18.3, based on kernel v1.36.3 # Serving database 'demo', using 16 threads # Compiled for x86_64-redhat-linux-gnu/64bit with 64bit OIDs dynamically linked # Found 47.259 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2010 MonetDB B.V., all rights reserved # Visit http://monetdb.cwi.nl/ for further information #warning: please don't forget to set your vault key! #(see /etc/monetdb5.conf) # Listening for connection requests on mapi:monetdb://127.0.0.1:50000/ # MonetDB/SQL module v2.36.3 loaded
\q
[root@monetdb masters]# mclient -lsql -d demo user(root):monetdb password: Welcome to mclient, the MonetDB/SQL interactive terminal Database: MonetDB v5.18.3, 'demo' Type \q to quit, \? for a list of available commands auto commit mode: on sql>copy select * from tables limit 10 into '/tmp/output.txt'; 10 affected rows sql>copy select * from plk100 limit 10 into '/tmp/output.txt'; MAPI = monetdb@localhost:50000 ACTION= read_line QUERY = copy select * from plk100 limit 10 into '/tmp/output.txt'; ERROR = !Connection terminated [root@monetdb masters]#
[root@monetdb MonetDB]# mserver5 --dbinit="include sql;" # MonetDB server v5.18.3, based on kernel v1.36.3 # Serving database 'demo', using 16 threads # Compiled for x86_64-redhat-linux-gnu/64bit with 64bit OIDs dynamically linked # Found 47.259 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2010 MonetDB B.V., all rights reserved # Visit http://monetdb.cwi.nl/ for further information #warning: please don't forget to set your vault key! #(see /etc/monetdb5.conf) # Listening for connection requests on mapi:monetdb://127.0.0.1:50000/ # MonetDB/SQL module v2.36.3 loaded
Segmentation fault
Thanks. Dariusz.
Stefan Manegold wrote:
On Wed, Apr 21, 2010 at 08:23:06PM +0200, Fabian Groffen wrote:
On 21-04-2010 14:15:33 -0400, dariuszs wrote:
> Single thread works, but what does it really mean? Thanks. Dariusz > > > It means you likely have the same problem as Hering Cheng, which seems related to parallelism.
You can try to unset (monetdb inherit nrthreads demo) the single threadedness once you have loaded the data.
Just to make sure, Dariusz' problem is not with loading data from .csv file into the database (copy into from file) but with dumping data from the database into a .csv file (copy select ... into file). Hence, is it not necessarily related to Hering Cheng's loading problem.
I just tried a (simplified) dump like
copy select * from tables limit 10 into '/tmp/output.txt';
on my 4-core Febdora 12 desktop with Feb2010, Feb2010-SP1 & Feb2010-SP2 preview, and all work fine.
Dariusz,
could you please test & report whether this simple version also works (in standard multi-threaded mode) also on your 16-core machine(s)?
Could you also check & report, wether the select part (without copy ... into file) works fine on your machine(s), i.e., select * from table1 limit 1000;
Thanks!
Stefan
------------------------------------------------------------------------------ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
------------------------------------------------------------------------------ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
------------------------------------------------------------------------------ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
Hi, I've got this in '/var/log/messages' after the crash Apr 22 15:40:52 localhost abrt[2247]: saved core dump of pid 2188 (/usr/bin/mserver5) to /var/cache/abrt/ccpp-1271965251-2188.new/coredump (159715328 bytes) Apr 22 15:40:52 localhost abrtd: Directory 'ccpp-1271965251-2188' creation detected Apr 22 15:40:52 localhost abrtd: Package 'MonetDB5-server' isn't signed with proper key Apr 22 15:40:52 localhost abrtd: Corrupted or bad crash /var/cache/abrt/ccpp-127965251-2188 (res:5), deleting Dariusz. Stefan Manegold wrote:
On Thu, Apr 22, 2010 at 11:34:53AM -0400, dariuszs wrote:
Hi, This is actually very good news, that means It's something my machine.
... or in your data, or in your compiler, or in your copy of the MonetDb source, or ...
If you don't mind - what's your hardware? Thanks. Dariusz.
No that I think it matters: Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz, 8 GB RAM
Stefan
Stefan Manegold wrote:
Hi,
I just tried your example, i.e., loading via copy into from file then dumping via copy select ... limit ... into file then restarting server and dumping again --- obviously not with your data, though, but with ~400000 rows of a 90+ column table, and that works fine with Feb2010, Feb2010-SP1 & Feb2010-SP2 preview on my 64-bit 4-core Fedora 12 desktop.
In case you are familiar with C development and debugging, you might want to compile a debugging version of MonetDB (configured with --enable-debug), run mserver5 in a debugger, locate the actual location of and reason for the segfault, and report. Remotely, I cannot do more since I cannot reproduce the segfault.
Stefan
On Thu, Apr 22, 2010 at 10:53:44AM -0400, dariuszs wrote:
Hi, OK, simple query works and there is something new - when I import data and run the query in the same session - it works, when I restart the server It segfaults - file permissions? Take a look.
[root@monetdb masters]# mclient -lsql -d demo user(root):monetdb password: Welcome to mclient, the MonetDB/SQL interactive terminal Database: MonetDB v5.18.3, 'demo' Type \q to quit, \? for a list of available commands auto commit mode: on sql>\< /vol2/monetdb_data/masters/create_plk100.sql operation successful sql>copy 1000000 records into plk100 from '/vol2/monetdb_data/masters/PLK100.DAT' delimiters '|','\n',''; 1000000 affected rows sql>copy select * from tables limit 10 into '/tmp/output.txt'; 10 affected rows sql>copy select * from tables limit 10 into '/tmp/output.txt'; 10 affected rows sql>copy select * from plk100 limit 10 into '/tmp/output.txt'; 10 affected rows sql>copy select * from plk100 limit 10 into '/tmp/output.txt'; 10 affected rows sql>\q
[root@monetdb MonetDB]# mserver5 --dbinit="include sql;" # MonetDB server v5.18.3, based on kernel v1.36.3 # Serving database 'demo', using 16 threads # Compiled for x86_64-redhat-linux-gnu/64bit with 64bit OIDs dynamically linked # Found 47.259 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2010 MonetDB B.V., all rights reserved # Visit http://monetdb.cwi.nl/ for further information #warning: please don't forget to set your vault key! #(see /etc/monetdb5.conf) # Listening for connection requests on mapi:monetdb://127.0.0.1:50000/ # MonetDB/SQL module v2.36.3 loaded
\q
[root@monetdb masters]# mclient -lsql -d demo user(root):monetdb password: Welcome to mclient, the MonetDB/SQL interactive terminal Database: MonetDB v5.18.3, 'demo' Type \q to quit, \? for a list of available commands auto commit mode: on sql>copy select * from tables limit 10 into '/tmp/output.txt'; 10 affected rows sql>copy select * from plk100 limit 10 into '/tmp/output.txt'; MAPI = monetdb@localhost:50000 ACTION= read_line QUERY = copy select * from plk100 limit 10 into '/tmp/output.txt'; ERROR = !Connection terminated [root@monetdb masters]#
[root@monetdb MonetDB]# mserver5 --dbinit="include sql;" # MonetDB server v5.18.3, based on kernel v1.36.3 # Serving database 'demo', using 16 threads # Compiled for x86_64-redhat-linux-gnu/64bit with 64bit OIDs dynamically linked # Found 47.259 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2010 MonetDB B.V., all rights reserved # Visit http://monetdb.cwi.nl/ for further information #warning: please don't forget to set your vault key! #(see /etc/monetdb5.conf) # Listening for connection requests on mapi:monetdb://127.0.0.1:50000/ # MonetDB/SQL module v2.36.3 loaded
Segmentation fault
Thanks. Dariusz.
Stefan Manegold wrote:
On Wed, Apr 21, 2010 at 08:23:06PM +0200, Fabian Groffen wrote:
On 21-04-2010 14:15:33 -0400, dariuszs wrote:
> Single thread works, but what does it really mean? Thanks. Dariusz > > > It means you likely have the same problem as Hering Cheng, which seems related to parallelism.
You can try to unset (monetdb inherit nrthreads demo) the single threadedness once you have loaded the data.
Just to make sure, Dariusz' problem is not with loading data from .csv file into the database (copy into from file) but with dumping data from the database into a .csv file (copy select ... into file). Hence, is it not necessarily related to Hering Cheng's loading problem.
I just tried a (simplified) dump like
copy select * from tables limit 10 into '/tmp/output.txt';
on my 4-core Febdora 12 desktop with Feb2010, Feb2010-SP1 & Feb2010-SP2 preview, and all work fine.
Dariusz,
could you please test & report whether this simple version also works (in standard multi-threaded mode) also on your 16-core machine(s)?
Could you also check & report, wether the select part (without copy ... into file) works fine on your machine(s), i.e., select * from table1 limit 1000;
Thanks!
Stefan
------------------------------------------------------------------------------ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
------------------------------------------------------------------------------ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
------------------------------------------------------------------------------ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
On Thu, Apr 22, 2010 at 03:42:52PM -0400, dariuszs wrote:
Hi, I've got this in '/var/log/messages' after the crash
Apr 22 15:40:52 localhost abrt[2247]: saved core dump of pid 2188 (/usr/bin/mserver5) to /var/cache/abrt/ccpp-1271965251-2188.new/coredump (159715328 bytes) Apr 22 15:40:52 localhost abrtd: Directory 'ccpp-1271965251-2188' creation detected Apr 22 15:40:52 localhost abrtd: Package 'MonetDB5-server' isn't signed with proper key
Our RPMs are (supposed to be) properly signed. How *exactly* did you install MonetDB? Stefan
Apr 22 15:40:52 localhost abrtd: Corrupted or bad crash /var/cache/abrt/ccpp-127965251-2188 (res:5), deleting
Dariusz.
Stefan Manegold wrote:
On Thu, Apr 22, 2010 at 11:34:53AM -0400, dariuszs wrote:
Hi, This is actually very good news, that means It's something my machine.
... or in your data, or in your compiler, or in your copy of the MonetDb source, or ...
If you don't mind - what's your hardware? Thanks. Dariusz.
No that I think it matters: Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz, 8 GB RAM
Stefan
Stefan Manegold wrote:
Hi,
I just tried your example, i.e., loading via copy into from file then dumping via copy select ... limit ... into file then restarting server and dumping again --- obviously not with your data, though, but with ~400000 rows of a 90+ column table, and that works fine with Feb2010, Feb2010-SP1 & Feb2010-SP2 preview on my 64-bit 4-core Fedora 12 desktop.
In case you are familiar with C development and debugging, you might want to compile a debugging version of MonetDB (configured with --enable-debug), run mserver5 in a debugger, locate the actual location of and reason for the segfault, and report. Remotely, I cannot do more since I cannot reproduce the segfault.
Stefan
On Thu, Apr 22, 2010 at 10:53:44AM -0400, dariuszs wrote:
Hi, OK, simple query works and there is something new - when I import data and run the query in the same session - it works, when I restart the server It segfaults - file permissions? Take a look.
[root@monetdb masters]# mclient -lsql -d demo user(root):monetdb password: Welcome to mclient, the MonetDB/SQL interactive terminal Database: MonetDB v5.18.3, 'demo' Type \q to quit, \? for a list of available commands auto commit mode: on sql>\< /vol2/monetdb_data/masters/create_plk100.sql operation successful sql>copy 1000000 records into plk100 from '/vol2/monetdb_data/masters/PLK100.DAT' delimiters '|','\n',''; 1000000 affected rows sql>copy select * from tables limit 10 into '/tmp/output.txt'; 10 affected rows sql>copy select * from tables limit 10 into '/tmp/output.txt'; 10 affected rows sql>copy select * from plk100 limit 10 into '/tmp/output.txt'; 10 affected rows sql>copy select * from plk100 limit 10 into '/tmp/output.txt'; 10 affected rows sql>\q
[root@monetdb MonetDB]# mserver5 --dbinit="include sql;" # MonetDB server v5.18.3, based on kernel v1.36.3 # Serving database 'demo', using 16 threads # Compiled for x86_64-redhat-linux-gnu/64bit with 64bit OIDs dynamically linked # Found 47.259 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2010 MonetDB B.V., all rights reserved # Visit http://monetdb.cwi.nl/ for further information #warning: please don't forget to set your vault key! #(see /etc/monetdb5.conf) # Listening for connection requests on mapi:monetdb://127.0.0.1:50000/ # MonetDB/SQL module v2.36.3 loaded
\q
[root@monetdb masters]# mclient -lsql -d demo user(root):monetdb password: Welcome to mclient, the MonetDB/SQL interactive terminal Database: MonetDB v5.18.3, 'demo' Type \q to quit, \? for a list of available commands auto commit mode: on sql>copy select * from tables limit 10 into '/tmp/output.txt'; 10 affected rows sql>copy select * from plk100 limit 10 into '/tmp/output.txt'; MAPI = monetdb@localhost:50000 ACTION= read_line QUERY = copy select * from plk100 limit 10 into '/tmp/output.txt'; ERROR = !Connection terminated [root@monetdb masters]#
[root@monetdb MonetDB]# mserver5 --dbinit="include sql;" # MonetDB server v5.18.3, based on kernel v1.36.3 # Serving database 'demo', using 16 threads # Compiled for x86_64-redhat-linux-gnu/64bit with 64bit OIDs dynamically linked # Found 47.259 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2010 MonetDB B.V., all rights reserved # Visit http://monetdb.cwi.nl/ for further information #warning: please don't forget to set your vault key! #(see /etc/monetdb5.conf) # Listening for connection requests on mapi:monetdb://127.0.0.1:50000/ # MonetDB/SQL module v2.36.3 loaded
Segmentation fault
Thanks. Dariusz.
Stefan Manegold wrote:
On Wed, Apr 21, 2010 at 08:23:06PM +0200, Fabian Groffen wrote:
> On 21-04-2010 14:15:33 -0400, dariuszs wrote: > > > >> Single thread works, but what does it really mean? Thanks. Dariusz >> >> >> > It means you likely have the same problem as Hering Cheng, which seems > related to parallelism. > > You can try to unset (monetdb inherit nrthreads demo) the single > threadedness once you have loaded the data. > > > Just to make sure, Dariusz' problem is not with loading data from .csv file into the database (copy into from file) but with dumping data from the database into a .csv file (copy select ... into file). Hence, is it not necessarily related to Hering Cheng's loading problem.
I just tried a (simplified) dump like
copy select * from tables limit 10 into '/tmp/output.txt';
on my 4-core Febdora 12 desktop with Feb2010, Feb2010-SP1 & Feb2010-SP2 preview, and all work fine.
Dariusz,
could you please test & report whether this simple version also works (in standard multi-threaded mode) also on your 16-core machine(s)?
Could you also check & report, wether the select part (without copy ... into file) works fine on your machine(s), i.e., select * from table1 limit 1000;
Thanks!
Stefan
> ------------------------------------------------------------------------------ > _______________________________________________ > MonetDB-users mailing list > MonetDB-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-users > > > > >
------------------------------------------------------------------------------ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
------------------------------------------------------------------------------ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
------------------------------------------------------------------------------ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
-- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4199 |
Hi, I've followed this Alternatively, download the file monetdb.repo http://monetdb.cwi.nl/downloads/Fedora/monetdb.repo and copy it to your /etc/yum.repos.d directory. You should also install the MonetDB GPG public key http://monetdb.cwi.nl/downloads/MonetDB-GPG-KEY: rpm --import http://monetdb.cwi.nl/downloads/MonetDB-GPG-KEY After this, you can use yum install MonetDB\* to install the MonetDB packages. Dariusz. Stefan Manegold wrote:
On Thu, Apr 22, 2010 at 03:42:52PM -0400, dariuszs wrote:
Hi, I've got this in '/var/log/messages' after the crash
Apr 22 15:40:52 localhost abrt[2247]: saved core dump of pid 2188 (/usr/bin/mserver5) to /var/cache/abrt/ccpp-1271965251-2188.new/coredump (159715328 bytes) Apr 22 15:40:52 localhost abrtd: Directory 'ccpp-1271965251-2188' creation detected Apr 22 15:40:52 localhost abrtd: Package 'MonetDB5-server' isn't signed with proper key
Our RPMs are (supposed to be) properly signed.
How *exactly* did you install MonetDB?
Stefan
Apr 22 15:40:52 localhost abrtd: Corrupted or bad crash /var/cache/abrt/ccpp-127965251-2188 (res:5), deleting
Dariusz.
Stefan Manegold wrote:
On Thu, Apr 22, 2010 at 11:34:53AM -0400, dariuszs wrote:
Hi, This is actually very good news, that means It's something my machine.
... or in your data, or in your compiler, or in your copy of the MonetDb source, or ...
If you don't mind - what's your hardware? Thanks. Dariusz.
No that I think it matters: Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz, 8 GB RAM
Stefan
Stefan Manegold wrote:
Hi,
I just tried your example, i.e., loading via copy into from file then dumping via copy select ... limit ... into file then restarting server and dumping again --- obviously not with your data, though, but with ~400000 rows of a 90+ column table, and that works fine with Feb2010, Feb2010-SP1 & Feb2010-SP2 preview on my 64-bit 4-core Fedora 12 desktop.
In case you are familiar with C development and debugging, you might want to compile a debugging version of MonetDB (configured with --enable-debug), run mserver5 in a debugger, locate the actual location of and reason for the segfault, and report. Remotely, I cannot do more since I cannot reproduce the segfault.
Stefan
On Thu, Apr 22, 2010 at 10:53:44AM -0400, dariuszs wrote:
Hi, OK, simple query works and there is something new - when I import data and run the query in the same session - it works, when I restart the server It segfaults - file permissions? Take a look.
[root@monetdb masters]# mclient -lsql -d demo user(root):monetdb password: Welcome to mclient, the MonetDB/SQL interactive terminal Database: MonetDB v5.18.3, 'demo' Type \q to quit, \? for a list of available commands auto commit mode: on sql>\< /vol2/monetdb_data/masters/create_plk100.sql operation successful sql>copy 1000000 records into plk100 from '/vol2/monetdb_data/masters/PLK100.DAT' delimiters '|','\n',''; 1000000 affected rows sql>copy select * from tables limit 10 into '/tmp/output.txt'; 10 affected rows sql>copy select * from tables limit 10 into '/tmp/output.txt'; 10 affected rows sql>copy select * from plk100 limit 10 into '/tmp/output.txt'; 10 affected rows sql>copy select * from plk100 limit 10 into '/tmp/output.txt'; 10 affected rows sql>\q
[root@monetdb MonetDB]# mserver5 --dbinit="include sql;" # MonetDB server v5.18.3, based on kernel v1.36.3 # Serving database 'demo', using 16 threads # Compiled for x86_64-redhat-linux-gnu/64bit with 64bit OIDs dynamically linked # Found 47.259 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2010 MonetDB B.V., all rights reserved # Visit http://monetdb.cwi.nl/ for further information #warning: please don't forget to set your vault key! #(see /etc/monetdb5.conf) # Listening for connection requests on mapi:monetdb://127.0.0.1:50000/ # MonetDB/SQL module v2.36.3 loaded >\q
[root@monetdb masters]# mclient -lsql -d demo user(root):monetdb password: Welcome to mclient, the MonetDB/SQL interactive terminal Database: MonetDB v5.18.3, 'demo' Type \q to quit, \? for a list of available commands auto commit mode: on sql>copy select * from tables limit 10 into '/tmp/output.txt'; 10 affected rows sql>copy select * from plk100 limit 10 into '/tmp/output.txt'; MAPI = monetdb@localhost:50000 ACTION= read_line QUERY = copy select * from plk100 limit 10 into '/tmp/output.txt'; ERROR = !Connection terminated [root@monetdb masters]#
[root@monetdb MonetDB]# mserver5 --dbinit="include sql;" # MonetDB server v5.18.3, based on kernel v1.36.3 # Serving database 'demo', using 16 threads # Compiled for x86_64-redhat-linux-gnu/64bit with 64bit OIDs dynamically linked # Found 47.259 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2010 MonetDB B.V., all rights reserved # Visit http://monetdb.cwi.nl/ for further information #warning: please don't forget to set your vault key! #(see /etc/monetdb5.conf) # Listening for connection requests on mapi:monetdb://127.0.0.1:50000/ # MonetDB/SQL module v2.36.3 loaded >Segmentation fault
Thanks. Dariusz.
Stefan Manegold wrote:
> On Wed, Apr 21, 2010 at 08:23:06PM +0200, Fabian Groffen wrote: > > > > >> On 21-04-2010 14:15:33 -0400, dariuszs wrote: >> >> >> >> >>> Single thread works, but what does it really mean? Thanks. Dariusz >>> >>> >>> >>> >> It means you likely have the same problem as Hering Cheng, which seems >> related to parallelism. >> >> You can try to unset (monetdb inherit nrthreads demo) the single >> threadedness once you have loaded the data. >> >> >> >> > Just to make sure, Dariusz' problem is not with loading data from .csv file > into the database (copy into from file) but with dumping data from the > database into a .csv file (copy select ... into file). > Hence, is it not necessarily related to Hering Cheng's loading problem. > > I just tried a (simplified) dump like > > copy select * from tables limit 10 into '/tmp/output.txt'; > > on my 4-core Febdora 12 desktop with Feb2010, Feb2010-SP1 & Feb2010-SP2 > preview, and all work fine. > > > Dariusz, > > could you please test & report whether this simple version also works (in > standard multi-threaded mode) also on your 16-core machine(s)? > > Could you also check & report, wether the select part (without copy ... into > file) works fine on your machine(s), i.e., > select * from table1 limit 1000; > > Thanks! > > Stefan > > > > > >> ------------------------------------------------------------------------------ >> _______________________________________________ >> MonetDB-users mailing list >> MonetDB-users@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/monetdb-users >> >> >> >> >> >> > > > > ------------------------------------------------------------------------------ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
------------------------------------------------------------------------------ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
------------------------------------------------------------------------------ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
Hi, select * from table1 limit 1000; works fine (without copy) Dariusz. Stefan Manegold wrote:
On Wed, Apr 21, 2010 at 08:23:06PM +0200, Fabian Groffen wrote:
On 21-04-2010 14:15:33 -0400, dariuszs wrote:
Single thread works, but what does it really mean? Thanks. Dariusz
It means you likely have the same problem as Hering Cheng, which seems related to parallelism.
You can try to unset (monetdb inherit nrthreads demo) the single threadedness once you have loaded the data.
Just to make sure, Dariusz' problem is not with loading data from .csv file into the database (copy into from file) but with dumping data from the database into a .csv file (copy select ... into file). Hence, is it not necessarily related to Hering Cheng's loading problem.
I just tried a (simplified) dump like
copy select * from tables limit 10 into '/tmp/output.txt';
on my 4-core Febdora 12 desktop with Feb2010, Feb2010-SP1 & Feb2010-SP2 preview, and all work fine.
Dariusz,
could you please test & report whether this simple version also works (in standard multi-threaded mode) also on your 16-core machine(s)?
Could you also check & report, wether the select part (without copy ... into file) works fine on your machine(s), i.e., select * from table1 limit 1000;
Thanks!
Stefan
------------------------------------------------------------------------------ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
Alternative method works with some hickups [root@monetdb MonetDB]# monetdb create demo created database in maintenance mode: demo [root@monetdb MonetDB]# monetdb set optpipe=inline,remap,deadcode,multiplex demo [root@monetdb MonetDB]# monetdb release demo taken database out of maintenance mode: demo [root@monetdb MonetDB]# mclient -lsql -d demo user(root):monetdb password: Welcome to mclient, the MonetDB/SQL interactive terminal Database: MonetDB v5.18.3, 'demo' Type \q to quit, \? for a list of available commands auto commit mode: on sql>\< /vol2/monetdb_data/masters/trw100_sample.sql auto commit mode: on 5000000 affected rows sql>copy select * from trw100 limit 10000 into '/tmp/f1.txt'; 10000 affected rows sql>\q [root@monetdb MonetDB]# mclient -lsql -d demo user(root):monetdb password: Connection terminated [root@monetdb MonetDB]# mclient -lsql -d demo user(root):monetdb password: Welcome to mclient, the MonetDB/SQL interactive terminal Database: MonetDB v5.18.3, 'demo' Type \q to quit, \? for a list of available commands auto commit mode: on sql>copy select * from trw100 limit 10000 into '/tmp/f1.txt'; 10000 affected rows sql> merovingian.log (partial) [root@monetdb MonetDB]# tail -n100 mero* 2010-04-21 14:35:13 ERR demo[12692]: 7f67c5220000-7f67c541f000 ---p 00004000 fd:00 396354 /usr/lib64/MonetDB5/lib/lib_blob.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c541f000-7f67c5420000 rw-p 00003000 fd:00 396354 /usr/lib64/MonetDB5/lib/lib_blob.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c5420000-7f67c52010-04-21 14:35:13 ERR demo[12692]: 425000 r-xp 00000000 fd:00 396441 /usr/lib64/MonetDB5/lib/lib_mkey.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c5425000-7f67c5624000 ---p 00005000 fd:00 396441 /usr/lib64/MonetDB5/lib/lib_mkey.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c5624000-7f67c5625000 rw-p 00004000 fd:00 396441 /usr/lib64/MonetDB5/lib/lib_mkey.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c5625000-7f67c566b000 r-xp 00000000 fd:00 396459 /usr/lib64/MonetDB5/lib/lib_pqueue.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c566b000-7f67c586a000 ---p 00046000 fd:00 396459 /usr/lib64/MonetDB5/lib/lib_pqueue.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c586a000-7f67c586b000 rw-p 00045000 fd:00 396459 /usr/lib64/MonetDB5/lib/lib_pqueue.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c586b000-7f67c5871000 r-xp 00000000 fd:00 396318 /usr/lib64/MonetDB5/lib/lib_array.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c5871000-7f67c5a70000 ---p 00006000 fd:00 396318 /usr/lib64/MonetDB5/lib/lib_array.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c5a70000-7f67c5a71000 rw-p 00005000 fd:00 396318 /usr/lib64/MonetDB5/lib/lib_array.2010-04-21 14:35:13 ERR demo[12692]: so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c5a71000-7f67c5e99000 r-xp 00000000 fd:00 396306 /usr/lib64/MonetDB5/lib/lib_aggr.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c5e99000-7f67c6099000 ---p 00428000 fd:00 396306 /usr/lib64/MonetDB5/lib/lib_aggr.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c6099000-7f67c609a000 rw-p 00428000 fd:00 396306 /usr/lib64/MonetDB5/lib/lib_aggr.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c609a000-7f67c615e000 r-xp 00000000 fd:00 396393 /usr/lib64/MonetDB5/lib/lib_group.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c615e000-7f67c635d000 ---p 000c4000 fd:00 396393 /usr/lib64/MonetDB5/lib/lib_group.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c635d000-7f67c635e000 rw-p 000c3000 fd:00 396393 /usr/lib64/MonetDB5/lib/lib_group.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c635e000-7f67c6363000 r-xp 00000000 fd:00 396501 /usr/lib64/MonetDB5/lib/lib_status.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c6363000-7f67c6562000 ---p 00005000 fd:00 396501 /usr/lib64/MonetDB5/lib/lib_status.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c6562000-7f67c6563000 rw-p 00004000 fd:00 396501 /usr/l2010-04-21 14:35:13 ERR demo[12692]: ib64/MonetDB5/lib/lib_status.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c6563000-7f67c658a000 r-xp 00000000 fd:00 396363 /usr/lib64/MonetDB5/lib/lib_calc.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c658a000-7f67c6789000 ---p 00027000 fd:00 396363 /usr/lib64/MonetDB5/lib/lib_calc.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c6789000-7f67c678a000 rw-p 00026000 fd:00 396363 /usr/lib64/MonetDB5/lib/lib_calc.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c678a000-7f67c678b000 r-xp 00000000 fd:00 396315 /usr/lib64/MonetDB5/lib/lib_algebraExtensions.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c678b000-7f67c698b000 ---p 00001000 fd:00 396315 /usr/lib64/MonetDB5/lib/lib_algebraExtensions.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c698b000-7f67c698c000 rw-p 00001000 fd:00 396315 /usr/lib64/MonetDB5/lib/lib_algebraExtensions.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c698c000-7f67c6990000 r-xp 00000000 fd:00 396324 /usr/lib64/MonetDB5/lib/lib_batExtensions.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c6990000-7f67c6b8f000 ---p 00004000 fd:00 396324 /usr/lib64/MonetDB5/lib/lib_batExtensions.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c6b8f000-7f67c6b90000 rw-p 00003000 fd:00 396324 /usr/lib64/MonetDB5/lib/lib_batExtensions.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c6b90000-7f67c6b93000 r-xp 00000000 fd:00 396507 /usr/lib64/MonetDB5/lib/lib_streams.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c6b93000-7f67c6d92000 ---p 00003000 fd:00 396507 /usr/lib64/MonetDB5/lib/lib_streams.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c6d92000-7f67c6d93000 rw-p 00002000 fd:00 396507 /usr/lib64/MonetDB5/lib/lib_streams.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c6d93000-7f67c6d98000 r-xp 00000000 fd:00 396444 /usr/lib64/MonetDB5/lib/lib_mmath.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c6d98000-7f67c6f97000 ---p 00005000 fd:00 396444 /usr/lib64/MonetDB5/lib/lib_mmath.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c6f97000-7f67c6f98000 rw-p 00004000 fd:00 396444 /usr/lib64/MonetDB5/lib/lib_mmath.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c6f98000-7f67c6f99000 r-xp 00000000 fd:00 396495 /usr/lib64/MonetDB5/lib/lib_sema.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c6f99000-7f67c7199000 ---p 00001000 fd:00 396495 /u2010-04-21 14:35:13 ERR demo[12692]: sr/lib64/MonetDB5/lib/lib_sema.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c7199000-7f67c719a000 rw-p 00001000 fd:00 396495 /usr/lib64/MonetDB5/lib/lib_sema.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c719a000-7f67c719c000 r-xp 00000000 fd:00 396414 /usr/lib64/MonetDB5/lib/lib_lock.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c719c000-7f67c739b000 ---p 00002000 fd:00 396414 /usr/lib64/MonetDB5/lib/lib_lock.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c739b000-7f67c739c000 rw-p 00001000 fd:00 396414 /usr/lib64/MonetDB5/lib/lib_lock.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c739c000-7f67c739d000 r-xp 00000000 fd:00 396522 /usr/lib64/MonetDB5/lib/lib_unix.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c739d000-7f67c759c000 ---p 00001000 fd:00 396522 /usr/lib64/MonetDB5/lib/lib_unix.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c759c000-7f67c759d000 rw-p 00000000 fd:00 396522 /usr/lib64/MonetDB5/lib/lib_unix.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c759d000-7f67c75e4000 r-xp 00000000 fd:00 396312 /usr/lib64/MonetDB5/lib/lib_algebra.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c75e4000-7f67c77e3000 ---p 00047000 fd:00 3963122010-04-21 14:35:13 ERR demo[12692]: /usr/lib64/MonetDB5/lib/lib_algebra.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c77e3000-7f67c77e4000 rw-p 00046000 fd:00 396312 /usr/lib64/MonetDB5/lib/lib_algebra.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c77e4000-7f67c77fb000 r-xp 00000000 fd:00 396321 /usr/lib64/MonetDB5/lib/lib_bat5.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c77fb000-7f67c79fb000 ---p 00017000 fd:00 396321 /usr/lib64/MonetDB5/lib/lib_bat5.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c79fb000-7f67c79fc000 rw-p 00017000 fd:00 396321 /usr/lib64/MonetDB5/lib/lib_bat5.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c79fc000-7f67c7a01000 r-xp 00000000 fd:00 396435 /usr/lib64/MonetDB5/lib/lib_mdb.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c7a01000-7f67c7c00000 ---p 00005000 fd:00 396435 /usr/lib64/MonetDB5/lib/lib_mdb.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c7c00000-7f67c7c01000 rw-p 00004000 fd:00 396435 /usr/lib64/MonetDB5/lib/lib_mdb.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67c7c01000-7f67c7c21000 rw-s 00000000 fd:00 6000 /var/MonetDB5/dbfarm/demo/bat/14/1414.theap 2010-04-21 14:35:13 ERR demo[12692]: 7f67c7c21000-7f67c7c41000 rw2010-04-21 14:35:13 ERR demo[12692]: -s 00000000 fd:00 6045 /var/MonetDB5/dbfarm/demo/bat/17/1700.theap 2010-04-21 14:35:13 ERR demo[12692]: 7f67c7c41000-7f67c7c61000 rw-s 00000000 fd:00 6044 /var/MonetDB5/dbfarm/demo/bat/16/1675.theap 2010-04-21 14:35:13 ERR demo[12692]: 7f67c7c61000-7f67c7c81000 rw-s 00000000 fd:00 6006 /var/MonetDB5/dbfarm/demo/bat/15/1522.theap 2010-04-21 14:35:13 ERR demo[12692]: 7f67c7c81000-7f67c7ca1000 rw-s 00000000 fd:00 5997 /var/MonetDB5/dbfarm/demo/bat/13/1374.theap 2010-04-21 14:35:13 ERR demo[12692]: 7f67c7ca1000-7f67c7cc1000 rw-s 00000000 fd:00 6038 /var/MonetDB5/dbfarm/demo/bat/17/1703.theap 2010-04-21 14:35:13 ERR demo[12692]: 7f67c7cc1000-7f67c7ce1000 rw-s 00000000 fd:00 6042 /var/MonetDB5/dbfarm/demo/bat/14/1410.theap 2010-04-21 14:35:13 ERR demo[12692]: 7f67c7ce1000-7f67c7d01000 rw-s 00000000 fd:00 6004 /var/MonetDB5/dbfarm/demo/bat/15/1500.theap 2010-04-21 14:35:13 ERR demo[12692]: 7f67c7d01000-7f67c7d02000 ---p 00000000 00:00 0 2010-04-21 14:35:13 ERR demo[12692]: 7f67c7d02000-7f67c7f2e000 rw-p 00000000 00:00 0 2010-04-21 14:35:13 ERR demo[12692]: 7f67c7f2e000-7f67cd702000 ---p 00000000 00:00 0 2010-04-21 14:35:13 ERR demo[12692]: 7f67cd702000-7f67d3533000 r--p 00000000 fd:00 263286 /usr/lib/loca2010-04-21 14:35:13 ERR demo[12692]: le/locale-archive 2010-04-21 14:35:13 ERR demo[12692]: 7f67d3533000-7f67d3539000 rw-p 00000000 00:00 0 2010-04-21 14:35:13 ERR demo[12692]: 7f67d3539000-7f67d3592000 r-xp 00000000 fd:00 263127 /usr/lib64/libfreebl3.so 2010-04-21 14:35:13 ERR demo[12692]: 7f67d3592000-7f67d3792000 ---p 00059000 fd:00 263127 /usr/lib64/libfreebl3.so 2010-04-21 14:35:13 ERR demo[12692]: 7f67d3792000-7f67d3793000 rw-p 00059000 fd:00 263127 /usr/lib64/libfreebl3.so 2010-04-21 14:35:13 ERR demo[12692]: 7f67d3793000-7f67d379b000 rw-p 00000000 00:00 0 2010-04-21 14:35:13 ERR demo[12692]: 7f67d379b000-7f67d37a2000 r-xp 00000000 fd:00 1477 /lib64/libcrypt-2.11.so 2010-04-21 14:35:13 ERR demo[12692]: 7f67d37a2000-7f67d39a2000 ---p 00007000 fd:00 1477 /lib64/libcrypt-2.11.so 2010-04-21 14:35:13 ERR demo[12692]: 7f67d39a2000-7f67d39a3000 r--p 00007000 fd:00 1477 /lib64/libcrypt-2.11.so 2010-04-21 14:35:13 ERR demo[12692]: 7f67d39a3000-7f67d39a4000 rw-p 00008000 fd:00 1477 /lib64/libcrypt-2.11.so 2010-04-21 14:35:13 ERR demo[12692]: 7f67d39a4000-7f67d39d6000 rw-p 00000000 00:00 0 2010-04-21 14:35:13 ERR demo[12692]: 7f67d39d6000-7f67d3a4c000 r-xp 00000000 fd:00 323652 /usr/lib64/libmonetdb5.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67d3a4c000-7f67d3c4c000 ---p 00076000 fd:00 323652 2010-04-21 14:35:13 ERR demo[12692]: /usr/lib64/libmonetdb5.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67d3c4c000-7f67d3c54000 rw-p 00076000 fd:00 323652 /usr/lib64/libmonetdb5.so.5.18.3 2010-04-21 14:35:13 ERR demo[12692]: 7f67d3c54000-7f67d3d01000 rw-p 00000000 00:00 0 2010-04-21 14:35:13 ERR demo[12692]: 7f67d3d0e000-7f67d3d10000 rw-p 00000000 00:00 0 2010-04-21 14:35:13 ERR demo[12692]: 7f67d3d10000-7f67d3d17000 r--s 00000000 fd:00 262948 /usr/lib64/gconv/gconv-modules.cache 2010-04-21 14:35:13 ERR demo[12692]: 7f67d3d17000-7f67d3d19000 rw-p 00000000 00:00 0 2010-04-21 14:35:13 ERR demo[12692]: 7fff58dc0000-7fff58dd5000 rw-p 00000000 00:00 0 [stack] 2010-04-21 14:35:13 ERR demo[12692]: 7fff58dff000-7fff58e00000 r-xp 00000000 00:00 0 [vdso] 2010-04-21 14:35:13 ERR demo[12692]: ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] 2010-04-21 14:35:13 MSG merovingian[12683]: client localhost:52907 has disconnected from proxy 2010-04-21 14:35:13 MSG merovingian[12683]: database 'demo' (12692) was killed by signal 6 2010-04-21 14:35:48 MSG merovingian[12683]: database 'demo' has crashed after start on 2010-04-21 14:32:37, attempting restart, up min/avg/max: 0s/0s/0s, crash average: 1.00 0.10 0.03 (1-0=1) 2010-04-21 14:35:49 MSG merovingian[12683]: proxying client localhost:52909 for database 'demo' to mapi:monetdb://127.0.0.1:50002/demo 2010-04-21 14:35:49 MSG demo[12723]: arguments: /usr/bin/mserver5 --config=/etc/monetdb5.conf --dbname=demo --dbinit=include sql; --set merovingian_uri=mapi:monetdb://monetdb:50000/demo --set monet_daemon=yes --set mapi_open=false --set mapi_autosense=true --set mapi_port=50001 --set monet_vault_key=/var/MonetDB5/dbfarm/demo/.vaultkey --set sql_optimizer=inline,remap,deadcode,multiplex 2010-04-21 14:35:49 MSG demo[12723]: # MonetDB server v5.18.3, based on kernel v1.36.3 2010-04-21 14:35:49 MSG demo[12723]: # Serving database 'demo', using 16 threads 2010-04-21 14:35:49 MSG demo[12723]: # Compiled for x86_64-redhat-linux-gnu/64bit with 64bit OIDs dynamically linked 2010-04-21 14:35:49 MSG demo[12723]: # Found 47.259 GiB available main-memory. 2010-04-21 14:35:49 MSG demo[12723]: # Copyright (c) 1993-July 2008 CWI. 2010-04-21 14:35:49 MSG demo[12723]: # Copyright (c) August 2008-2010 MonetDB B.V., all rights reserved 2010-04-21 14:35:49 MSG demo[12723]: # Visit http://monetdb.cwi.nl/ for further information 2010-04-21 14:35:49 MSG demo[12723]: # Listening for connection requests on mapi:monetdb://127.0.0.1:50002/ 2010-04-21 14:35:49 MSG demo[12723]: # MonetDB/SQL module v2.36.3 loaded 2010-04-21 14:36:10 MSG merovingian[12683]: client localhost:52909 has disconnected from proxy dariuszs wrote:
Hi, Single thread works, but what does it really mean? Thanks. Dariusz
Fabian Groffen wrote:
On 21-04-2010 13:51:52 -0400, dariuszs wrote:
Hi, I'm using Fedora 12 on this machine, are you saying you don't have this problem? I've replicated this problem also on Ubuntu 9.10 and Windows 2003. Thanks. Dariusz.
Can you try the following to see if it makes any difference?
monetdb destroy demo monetdb create demo monetdb set nrthreads=1 demo monetdb release demo [load data]
Alternatively try this (but I think it has very little chance of making any difference):
monetdb destroy demo monetdb create demo monetdb set optpipe=inline,remap,deadcode,multiplex demo monetdb release demo [load data]
------------------------------------------------------------------------------ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
------------------------------------------------------------------------------ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
Hi, This used to work in Feb2009 release on the same hardware and that's what confuses me. Thanks. Dariusz. Fabian Groffen wrote:
On 21-04-2010 14:39:48 -0400, dariuszs wrote:
Alternative method works with some hickups
Seems you get a glibc double free abort. Perhaps it's worth waiting for Feb2010-SP2 to see how that one behaves under your load.
------------------------------------------------------------------------------ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
Hi, What are the downfalls of running in single thread mode? Thanks. Dariusz. Fabian Groffen wrote:
On 21-04-2010 14:39:48 -0400, dariuszs wrote:
Alternative method works with some hickups
Seems you get a glibc double free abort. Perhaps it's worth waiting for Feb2010-SP2 to see how that one behaves under your load.
------------------------------------------------------------------------------ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
dariuszs wrote:
Hi, Single thread works, but what does it really mean? Thanks. Dariusz That there is just one MAL interpreter active and in the copy code a single worker to process the tokenized intermediates.
Fabian Groffen wrote:
On 21-04-2010 13:51:52 -0400, dariuszs wrote:
Hi, I'm using Fedora 12 on this machine, are you saying you don't have this problem? I've replicated this problem also on Ubuntu 9.10 and Windows 2003. Thanks. Dariusz.
Can you try the following to see if it makes any difference?
monetdb destroy demo monetdb create demo monetdb set nrthreads=1 demo monetdb release demo [load data]
Alternatively try this (but I think it has very little chance of making any difference):
monetdb destroy demo monetdb create demo monetdb set optpipe=inline,remap,deadcode,multiplex demo monetdb release demo [load data]
------------------------------------------------------------------------------ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
------------------------------------------------------------------------------ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
[root@monetdb MonetDB]# more mero* 2010-04-21 13:21:12 MSG merovingian[12373]: Merovingian 1.2 starting 2010-04-21 13:21:12 MSG merovingian[12373]: monitoring dbfarm /var/MonetDB5/db farm 2010-04-21 13:21:12 MSG merovingian[12373]: listening for TCP connections on 0 .0.0.0:50000 2010-04-21 13:21:12 MSG discovery[12373]: listening for UDP messages on 0.0.0. 0:50000 2010-04-21 13:21:12 MSG control[12373]: handling commands over UNIX socket /va r/MonetDB5/dbfarm/.merovingian_control 2010-04-21 13:21:12 MSG control[12373]: listening for TCP connections on 0.0.0 .0:50001 2010-04-21 13:21:16 MSG control[12373]: (local): created database 'demo' 2010-04-21 13:21:21 MSG control[12373]: (local): released database 'demo' 2010-04-21 13:21:33 MSG merovingian[12373]: starting database 'demo', up min/a vg/max: 0s/0s/0s, crash average: 0.00 0.00 0.00 (0-0=0) 2010-04-21 13:21:33 MSG merovingian[12373]: proxying client localhost:44278 fo r database 'demo' to mapi:monetdb://127.0.0.1:50002/demo 2010-04-21 13:21:33 MSG demo[12380]: arguments: /usr/bin/mserver5 --config=/et c/monetdb5.conf --dbname=demo --dbinit=include sql; --set merovingian_uri=mapi :monetdb://monetdb:50000/demo --set monet_daemon=yes --set mapi_open=false --s et mapi_autosense=true --set mapi_port=50001 --set monet_vault_key=/var/MonetD B5/dbfarm/demo/.vaultkey --set sql_optimizer=default_pipe 2010-04-21 13:21:33 MSG demo[12380]: # MonetDB server v5.18.3, based on kernel v1.36.3 2010-04-21 13:21:33 MSG demo[12380]: # Serving database 'demo', using 16 threa ds 2010-04-21 13:21:33 MSG demo[12380]: # Compiled for x86_64-redhat-linux-gnu/64 bit with 64bit OIDs dynamically linked 2010-04-21 13:21:33 MSG demo[12380]: # Found 47.259 GiB available main-memory. 2010-04-21 13:21:33 MSG demo[12380]: # Copyright (c) 1993-July 2008 CWI. 2010-04-21 13:21:33 MSG demo[12380]: # Copyright (c) August 2008-2010 MonetDB B.V., all rights reserved 2010-04-21 13:21:33 MSG demo[12380]: # Visit http://monetdb.cwi.nl/ for furthe r information 2010-04-21 13:21:33 MSG demo[12380]: # Listening for connection requests on ma pi:monetdb://127.0.0.1:50002/ 2010-04-21 13:21:33 MSG demo[12380]: # MonetDB/SQL module v2.36.3 loaded 2010-04-21 13:24:01 MSG merovingian[12373]: client localhost:44278 has disconn ected from proxy 2010-04-21 13:24:07 MSG merovingian[12373]: proxying client localhost:53092 fo r database 'demo' to mapi:monetdb://127.0.0.1:50002/demo 2010-04-21 13:24:21 MSG merovingian[12373]: client localhost:53092 has disconn ected from proxy 2010-04-21 13:24:21 MSG merovingian[12373]: database 'demo' (12380) was killed by signal 11 [root@monetdb MonetDB]# (using merovingian) Thanks. Dariusz. Fabian Groffen wrote:
On 21-04-2010 13:07:37 -0400, dariuszs wrote:
Hi, It looks like the same problem exists no matter what the OS (Fedora 12 or Ubuntu 9.10 or Windows) and this worked in Nov2009-SP1 release. When you try: sql>copy select * from table1 limit 1000 into '/tmp/output.txt'; MAPI = monetdb@localhost:50000 ACTION= read_line QUERY = copy select * from table1 limit 1000 into '/tmp/output.txt'; ERROR = !Connection terminated Timer 1105.876 msec
please check your merovingian.log or provide us with the console output of the mserver5 process.
[root@monetdb /]# mserver5 --version MonetDB server v5.18.3 (64-bit), based on kernel v1.36.3 (64-bit oids) Copyright (c) 1993-July 2008 CWI Copyright (c) August 2008-2010 MonetDB B.V., all rights reserved Visit http://monetdb.cwi.nl/ for further information Found 47.3GiB available memory, 16 available cpu cores Configured for prefix: /usr Libraries: libpcre: 7.8 2008-09-05 (compiled with 7.8) openssl: OpenSSL 1.0.0-beta4 10 Nov 2009 (compiled with OpenSSL 1.0.0-fips-beta4 10 Nov 2009) libxml2: 2.7.6 (compiled with 2.7.6) Compiled by: mockbuild@ (x86_64-redhat-linux-gnu) Compilation: gcc -O2 -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -O6 -fomit-frame-pointer -finline-functions -falign-loops=4 -falign-jumps=4 -falign-functions=4 -fexpensive-optimizations -funroll-loops -frerun-cse-after-loop -frerun-loop-opt -ftree-vectorize Linking : ld -IPA -m elf_x86_64
------------------------------------------------------------------------------ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
participants (4)
-
dariuszs -
Fabian Groffen -
Martin Kersten -
Stefan Manegold