[MonetDB-users] Version 5 crashing.
My monetdb has been constantly crashing in the past several days and I am not able to even get to the SQL prompt of my database. Its just hanging Some information MSG DataBase[29614]: # Compiled for x86_64-unknown-linux-gnu/64bit with 64bit OIDs dynamically linked -bash-3.2$ /home/dbuser/MonetDB/bin/monetdb status name state uptime health last crash DataBase running 2h 34m 33s 40%, 2h 2009-05-12 16:49:50 demo stopped 100%, 4s - DataBase.1 stopped 100%, 2h - -bash-3.2$ /home/dbuser/MonetDB/bin/monetdb version MonetDB Database Server Toolkit v0.4 MSG merovingian[18807]: database 'DataBase' already running since 2009-05-12 16:52:20, up min/avg/max: 8609/8609/8609, crash average: 0.00 0.30 0.10 (5-1=3) MSG merovingian[18807]: proxying client 127.0.0.1:38462 for database 'DataBase' to mapi:monetdb://server01:50001/ MSG merovingian[18807]: client has 127.0.0.1:38462 disconnected from proxy Anything I should do to fix this? TIA
I also get,
!SQLException:SQLinit:Catalogue initialization failed
When I try to run, "mclient -lsql --database=DataBase
Any suggestions? I am getting desperate. I don't want to reload this
data it takes several hours to do :-(
On Tue, May 12, 2009 at 4:37 PM, Mag Gam
My monetdb has been constantly crashing in the past several days and I am not able to even get to the SQL prompt of my database. Its just hanging
Some information MSG DataBase[29614]: # Compiled for x86_64-unknown-linux-gnu/64bit with 64bit OIDs dynamically linked
-bash-3.2$ /home/dbuser/MonetDB/bin/monetdb status name state uptime health last crash DataBase running 2h 34m 33s 40%, 2h 2009-05-12 16:49:50 demo stopped 100%, 4s - DataBase.1 stopped 100%, 2h -
-bash-3.2$ /home/dbuser/MonetDB/bin/monetdb version MonetDB Database Server Toolkit v0.4
MSG merovingian[18807]: database 'DataBase' already running since 2009-05-12 16:52:20, up min/avg/max: 8609/8609/8609, crash average: 0.00 0.30 0.10 (5-1=3) MSG merovingian[18807]: proxying client 127.0.0.1:38462 for database 'DataBase' to mapi:monetdb://server01:50001/ MSG merovingian[18807]: client has 127.0.0.1:38462 disconnected from proxy
Anything I should do to fix this?
TIA
More information about the server:
MonetDB server v5.10.4 (64-bit), based on kernel v1.28.4 (64-bit oids)
Copyright (c) 1993-July 2008 CWI
Copyright (c) August 2008-2009 MonetDB B.V., all rights reserved
Visit http://monetdb.cwi.nl/ for further information
Libraries:
libpcre: 6.6 06-Feb-2006 (compiled with 6.6)
openssl: OpenSSL 0.9.8b 04 May 2006 (compiled with OpenSSL 0.9.8b 04 May 2006)
libxml2: 2.6.26 (compiled with 2.6.26)
Compilation: gcc -O2 -std=c99
Linking : ld -IPA -m elf_x86_64
Also,
${prefix}/var/MonetDB5/dbfarm exists on a network filesystem. I am
hoping thats ok (with flocking)
On Wed, May 13, 2009 at 4:14 AM, Mag Gam
I also get,
!SQLException:SQLinit:Catalogue initialization failed
When I try to run, "mclient -lsql --database=DataBase
Any suggestions? I am getting desperate. I don't want to reload this data it takes several hours to do :-(
On Tue, May 12, 2009 at 4:37 PM, Mag Gam
wrote: My monetdb has been constantly crashing in the past several days and I am not able to even get to the SQL prompt of my database. Its just hanging
Some information MSG DataBase[29614]: # Compiled for x86_64-unknown-linux-gnu/64bit with 64bit OIDs dynamically linked
-bash-3.2$ /home/dbuser/MonetDB/bin/monetdb status name state uptime health last crash DataBase running 2h 34m 33s 40%, 2h 2009-05-12 16:49:50 demo stopped 100%, 4s - DataBase.1 stopped 100%, 2h -
-bash-3.2$ /home/dbuser/MonetDB/bin/monetdb version MonetDB Database Server Toolkit v0.4
MSG merovingian[18807]: database 'DataBase' already running since 2009-05-12 16:52:20, up min/avg/max: 8609/8609/8609, crash average: 0.00 0.30 0.10 (5-1=3) MSG merovingian[18807]: proxying client 127.0.0.1:38462 for database 'DataBase' to mapi:monetdb://server01:50001/ MSG merovingian[18807]: client has 127.0.0.1:38462 disconnected from proxy
Anything I should do to fix this?
TIA
On 13-05-2009 04:34:04 -0700, Mag Gam wrote:
More information about the server:
MonetDB server v5.10.4 (64-bit), based on kernel v1.28.4 (64-bit oids) Copyright (c) 1993-July 2008 CWI Copyright (c) August 2008-2009 MonetDB B.V., all rights reserved Visit http://monetdb.cwi.nl/ for further information
Libraries: libpcre: 6.6 06-Feb-2006 (compiled with 6.6) openssl: OpenSSL 0.9.8b 04 May 2006 (compiled with OpenSSL 0.9.8b 04 May 2006) libxml2: 2.6.26 (compiled with 2.6.26)
Compilation: gcc -O2 -std=c99 Linking : ld -IPA -m elf_x86_64
Also,
${prefix}/var/MonetDB5/dbfarm exists on a network filesystem. I am hoping thats ok (with flocking)
Ohhh... Never tested, but that may be (a lot) slower... Do you really have to be on a network share? For speed it is regardless a good idea to have the storage local. mmapping on network shares might also be a bit less optimal.
Yes, have to be on a network filesystem. Its pretty fast, I can get
close to 500MB/sec consistently. Also, it has over 50TB of stable
available :-)
This issue is getting very frustrating. :-(
On Wed, May 13, 2009 at 7:49 AM, Fabian Groffen
On 13-05-2009 04:34:04 -0700, Mag Gam wrote:
More information about the server:
MonetDB server v5.10.4 (64-bit), based on kernel v1.28.4 (64-bit oids) Copyright (c) 1993-July 2008 CWI Copyright (c) August 2008-2009 MonetDB B.V., all rights reserved Visit http://monetdb.cwi.nl/ for further information
Libraries: libpcre: 6.6 06-Feb-2006 (compiled with 6.6) openssl: OpenSSL 0.9.8b 04 May 2006 (compiled with OpenSSL 0.9.8b 04 May 2006) libxml2: 2.6.26 (compiled with 2.6.26)
Compilation: gcc -O2 -std=c99 Linking : ld -IPA -m elf_x86_64
Also,
${prefix}/var/MonetDB5/dbfarm exists on a network filesystem. I am hoping thats ok (with flocking)
Ohhh... Never tested, but that may be (a lot) slower... Do you really have to be on a network share? For speed it is regardless a good idea to have the storage local. mmapping on network shares might also be a bit less optimal.
------------------------------------------------------------------------------ The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your production scanning environment may not be a perfect world - but thanks to Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700 Series Scanner you'll get full speed at 300 dpi even with all image processing features enabled. http://p.sf.net/sfu/kodak-com _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
Databases on NFS are not supported and lead to crashes on a mmaped
dbfarm afaik.
On May 13, 2009, at 2:13 PM, "Mag Gam"
Yes, have to be on a network filesystem. Its pretty fast, I can get close to 500MB/sec consistently. Also, it has over 50TB of stable available :-)
This issue is getting very frustrating. :-(
On Wed, May 13, 2009 at 7:49 AM, Fabian Groffen
wrote: On 13-05-2009 04:34:04 -0700, Mag Gam wrote:
More information about the server:
MonetDB server v5.10.4 (64-bit), based on kernel v1.28.4 (64-bit oids) Copyright (c) 1993-July 2008 CWI Copyright (c) August 2008-2009 MonetDB B.V., all rights reserved Visit http://monetdb.cwi.nl/ for further information
Libraries: libpcre: 6.6 06-Feb-2006 (compiled with 6.6) openssl: OpenSSL 0.9.8b 04 May 2006 (compiled with OpenSSL 0.9.8b 04 May 2006) libxml2: 2.6.26 (compiled with 2.6.26)
Compilation: gcc -O2 -std=c99 Linking : ld -IPA -m elf_x86_64
Also,
${prefix}/var/MonetDB5/dbfarm exists on a network filesystem. I am hoping thats ok (with flocking)
Ohhh... Never tested, but that may be (a lot) slower... Do you really have to be on a network share? For speed it is regardless a good idea to have the storage local. mmapping on network shares might also be a bit less optimal.
--- --- --- --------------------------------------------------------------------- The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your production scanning environment may not be a perfect world - but thanks to Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700 Series Scanner you'll get full speed at 300 dpi even with all image processing features enabled. http://p.sf.net/sfu/kodak-com _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
--- --- --- --------------------------------------------------------------------- The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your production scanning environment may not be a perfect world - but thanks to Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700 Series Scanner you'll get full speed at 300 dpi even with all image processing features enabled. http://p.sf.net/sfu/kodak-com _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
Thanks for the response Peter. This is on the Lustre filesystem.
http://arch.lustre.org/images/9/95/Mmap_hld.pdf
I believe it supports it without any issues. Or could this be an issue?
On Wed, May 13, 2009 at 8:29 AM, Peter Boncz
Databases on NFS are not supported and lead to crashes on a mmaped dbfarm afaik.
On May 13, 2009, at 2:13 PM, "Mag Gam"
wrote: Yes, have to be on a network filesystem. Its pretty fast, I can get close to 500MB/sec consistently. Also, it has over 50TB of stable available :-)
This issue is getting very frustrating. :-(
On Wed, May 13, 2009 at 7:49 AM, Fabian Groffen
wrote: On 13-05-2009 04:34:04 -0700, Mag Gam wrote:
More information about the server:
MonetDB server v5.10.4 (64-bit), based on kernel v1.28.4 (64-bit oids) Copyright (c) 1993-July 2008 CWI Copyright (c) August 2008-2009 MonetDB B.V., all rights reserved Visit http://monetdb.cwi.nl/ for further information
Libraries: libpcre: 6.6 06-Feb-2006 (compiled with 6.6) openssl: OpenSSL 0.9.8b 04 May 2006 (compiled with OpenSSL 0.9.8b 04 May 2006) libxml2: 2.6.26 (compiled with 2.6.26)
Compilation: gcc -O2 -std=c99 Linking : ld -IPA -m elf_x86_64
Also,
${prefix}/var/MonetDB5/dbfarm exists on a network filesystem. I am hoping thats ok (with flocking)
Ohhh... Never tested, but that may be (a lot) slower... Do you really have to be on a network share? For speed it is regardless a good idea to have the storage local. mmapping on network shares might also be a bit less optimal.
--- --- --- --------------------------------------------------------------------- The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your production scanning environment may not be a perfect world - but thanks to Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700 Series Scanner you'll get full speed at 300 dpi even with all image processing features enabled. http://p.sf.net/sfu/kodak-com _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
--- --- --- --------------------------------------------------------------------- The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your production scanning environment may not be a perfect world - but thanks to Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700 Series Scanner you'll get full speed at 300 dpi even with all image processing features enabled. http://p.sf.net/sfu/kodak-com _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
I have never heard of Lustre and I am not aware of anyone using that
with MonetDB.. A few years ago MonetDB still worked with (nfs) mmapped
filed, but it was slow and their were consistency risks as dirtypages
are not synced immediate over the network. For these reasons nobody
used it. Then, it stopped working. We should clean up the crash and
just make it fail or warn gracefully.
MonetDB does pretty scary stuff with mmap so without studying Lustre I
cannot say wheter it could work.
On May 13, 2009, at 2:32 PM, "Mag Gam"
Thanks for the response Peter. This is on the Lustre filesystem. http://arch.lustre.org/images/9/95/Mmap_hld.pdf
I believe it supports it without any issues. Or could this be an issue?
On Wed, May 13, 2009 at 8:29 AM, Peter Boncz
wrote: Databases on NFS are not supported and lead to crashes on a mmaped dbfarm afaik.
On May 13, 2009, at 2:13 PM, "Mag Gam"
wrote: Yes, have to be on a network filesystem. Its pretty fast, I can get close to 500MB/sec consistently. Also, it has over 50TB of stable available :-)
This issue is getting very frustrating. :-(
On Wed, May 13, 2009 at 7:49 AM, Fabian Groffen
wrote: On 13-05-2009 04:34:04 -0700, Mag Gam wrote:
More information about the server:
MonetDB server v5.10.4 (64-bit), based on kernel v1.28.4 (64-bit oids) Copyright (c) 1993-July 2008 CWI Copyright (c) August 2008-2009 MonetDB B.V., all rights reserved Visit http://monetdb.cwi.nl/ for further information
Libraries: libpcre: 6.6 06-Feb-2006 (compiled with 6.6) openssl: OpenSSL 0.9.8b 04 May 2006 (compiled with OpenSSL 0.9.8b 04 May 2006) libxml2: 2.6.26 (compiled with 2.6.26)
Compilation: gcc -O2 -std=c99 Linking : ld -IPA -m elf_x86_64
Also,
${prefix}/var/MonetDB5/dbfarm exists on a network filesystem. I am hoping thats ok (with flocking)
Ohhh... Never tested, but that may be (a lot) slower... Do you really have to be on a network share? For speed it is regardless a good idea to have the storage local. mmapping on network shares might also be a bit less optimal.
--- --- --- --- ------------------------------------------------------------------ The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your production scanning environment may not be a perfect world - but thanks to Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700 Series Scanner you'll get full speed at 300 dpi even with all image processing features enabled. http://p.sf.net/sfu/kodak-com _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
--- --- --- --- ------------------------------------------------------------------ The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your production scanning environment may not be a perfect world - but thanks to Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700 Series Scanner you'll get full speed at 300 dpi even with all image processing features enabled. http://p.sf.net/sfu/kodak-com _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
Ok. I spoke to several people at Lustre and they say mmap() is
implemented properly on Lustre and any application that uses mmap()
should work.
Is there a way to simulate a mmap() call (I am not an expert
programmer). I would like to see if this is a legemit bug on MonetDB
or Lustre.
Would anyone like to help me with this?
TIA
On Wed, May 13, 2009 at 8:59 AM, Peter Boncz
I have never heard of Lustre and I am not aware of anyone using that with MonetDB.. A few years ago MonetDB still worked with (nfs) mmapped filed, but it was slow and their were consistency risks as dirtypages are not synced immediate over the network. For these reasons nobody used it. Then, it stopped working. We should clean up the crash and just make it fail or warn gracefully.
MonetDB does pretty scary stuff with mmap so without studying Lustre I cannot say wheter it could work.
On May 13, 2009, at 2:32 PM, "Mag Gam"
wrote: Thanks for the response Peter. This is on the Lustre filesystem. http://arch.lustre.org/images/9/95/Mmap_hld.pdf
I believe it supports it without any issues. Or could this be an issue?
On Wed, May 13, 2009 at 8:29 AM, Peter Boncz
wrote: Databases on NFS are not supported and lead to crashes on a mmaped dbfarm afaik.
On May 13, 2009, at 2:13 PM, "Mag Gam"
wrote: Yes, have to be on a network filesystem. Its pretty fast, I can get close to 500MB/sec consistently. Also, it has over 50TB of stable available :-)
This issue is getting very frustrating. :-(
On Wed, May 13, 2009 at 7:49 AM, Fabian Groffen
wrote: On 13-05-2009 04:34:04 -0700, Mag Gam wrote:
More information about the server:
MonetDB server v5.10.4 (64-bit), based on kernel v1.28.4 (64-bit oids) Copyright (c) 1993-July 2008 CWI Copyright (c) August 2008-2009 MonetDB B.V., all rights reserved Visit http://monetdb.cwi.nl/ for further information
Libraries: libpcre: 6.6 06-Feb-2006 (compiled with 6.6) openssl: OpenSSL 0.9.8b 04 May 2006 (compiled with OpenSSL 0.9.8b 04 May 2006) libxml2: 2.6.26 (compiled with 2.6.26)
Compilation: gcc -O2 -std=c99 Linking : ld -IPA -m elf_x86_64
Also,
${prefix}/var/MonetDB5/dbfarm exists on a network filesystem. I am hoping thats ok (with flocking)
Ohhh... Never tested, but that may be (a lot) slower... Do you really have to be on a network share? For speed it is regardless a good idea to have the storage local. mmapping on network shares might also be a bit less optimal.
--- --- --- --- ------------------------------------------------------------------ The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your production scanning environment may not be a perfect world - but thanks to Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700 Series Scanner you'll get full speed at 300 dpi even with all image processing features enabled. http://p.sf.net/sfu/kodak-com _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
--- --- --- --- ------------------------------------------------------------------ The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your production scanning environment may not be a perfect world - but thanks to Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700 Series Scanner you'll get full speed at 300 dpi even with all image processing features enabled. http://p.sf.net/sfu/kodak-com _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
Could you provide an strace (start the server with "strace" in front
of the command)?
On May 14, 2009, at 3:03 PM, "Mag Gam"
Ok. I spoke to several people at Lustre and they say mmap() is implemented properly on Lustre and any application that uses mmap() should work.
Is there a way to simulate a mmap() call (I am not an expert programmer). I would like to see if this is a legemit bug on MonetDB or Lustre.
Would anyone like to help me with this?
TIA
On Wed, May 13, 2009 at 8:59 AM, Peter Boncz
wrote: I have never heard of Lustre and I am not aware of anyone using that with MonetDB.. A few years ago MonetDB still worked with (nfs) mmapped filed, but it was slow and their were consistency risks as dirtypages are not synced immediate over the network. For these reasons nobody used it. Then, it stopped working. We should clean up the crash and just make it fail or warn gracefully.
MonetDB does pretty scary stuff with mmap so without studying Lustre I cannot say wheter it could work.
On May 13, 2009, at 2:32 PM, "Mag Gam"
wrote: Thanks for the response Peter. This is on the Lustre filesystem. http://arch.lustre.org/images/9/95/Mmap_hld.pdf
I believe it supports it without any issues. Or could this be an issue?
On Wed, May 13, 2009 at 8:29 AM, Peter Boncz
wrote: Databases on NFS are not supported and lead to crashes on a mmaped dbfarm afaik.
On May 13, 2009, at 2:13 PM, "Mag Gam"
wrote: Yes, have to be on a network filesystem. Its pretty fast, I can get close to 500MB/sec consistently. Also, it has over 50TB of stable available :-)
This issue is getting very frustrating. :-(
On Wed, May 13, 2009 at 7:49 AM, Fabian Groffen
wrote: On 13-05-2009 04:34:04 -0700, Mag Gam wrote: > More information about the server: > > > MonetDB server v5.10.4 (64-bit), based on kernel v1.28.4 (64-bit > oids) > Copyright (c) 1993-July 2008 CWI > Copyright (c) August 2008-2009 MonetDB B.V., all rights reserved > Visit http://monetdb.cwi.nl/ for further information > > > Libraries: > libpcre: 6.6 06-Feb-2006 (compiled with 6.6) > openssl: OpenSSL 0.9.8b 04 May 2006 (compiled with OpenSSL > 0.9.8b 04 May 2006) > libxml2: 2.6.26 (compiled with 2.6.26) > > Compilation: gcc -O2 -std=c99 > Linking : ld -IPA -m elf_x86_64 > > Also, > > ${prefix}/var/MonetDB5/dbfarm exists on a network filesystem. > I am > hoping thats ok (with flocking)
Ohhh... Never tested, but that may be (a lot) slower... Do you really have to be on a network share? For speed it is regardless a good idea to have the storage local. mmapping on network shares might also be a bit less optimal.
--- --- --- --- --- --------------------------------------------------------------- The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your production scanning environment may not be a perfect world - but thanks to Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700 Series Scanner you'll get full speed at 300 dpi even with all image processing features enabled. http://p.sf.net/sfu/kodak-com _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
--- --- --- --- ------------------------------------------------------------------ The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your production scanning environment may not be a perfect world - but thanks to Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700 Series Scanner you'll get full speed at 300 dpi even with all image processing features enabled. http://p.sf.net/sfu/kodak-com _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
strace on the process (mserver) just gives:
nanosleep({5, 0}, {5, 0}) = 0
nanosleep({5, 0}, {5, 0}) = 0
nanosleep({5, 0}, {5, 0}) = 0
nanosleep({5, 0}, {5, 0}) = 0
nanosleep({5, 0}, {5, 0}) = 0
nanosleep({5, 0}, {5, 0}) = 0
strace on merovrian give me, when I run mclient -l sql dbname:
read(16, "", 2097152) = 0
close(16) = 0
munmap(0x2aaaaaeca000, 2097152) = 0
close(14) = 0
stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=3519, ...}) = 0
write(1, "database \'DataBase\' already running s"..., 128) = 128
getpeername(12, {sa_family=AF_INET, sin_port=htons(41314),
sin_addr=inet_addr("127.0.0.1")}, [16]) = 0
write(1, "proxying client 127.0.0.1:41314 "..., 89) = 89
write(12, "1\0", 2) = 2
write(12, "^mapi:merovingian:proxy\n", 24) = 24
open("/etc/hosts", O_RDONLY) = 14
fcntl(14, F_GETFD) = 0
fcntl(14, F_SETFD, FD_CLOEXEC) = 0
fstat(14, {st_mode=S_IFREG|0644, st_size=306, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = 0x2aaaaaeca000
read(14, "127.0.0.1\t\tlocalhost\n141.128.90."..., 4096) = 306
close(14) = 0
munmap(0x2aaaaaeca000, 4096) = 0
socket(PF_INET, SOCK_STREAM, IPPROTO_TCP) = 14
connect(14, {sa_family=AF_INET, sin_port=htons(50001),
sin_addr=inet_addr("141.128.90.133")}, 16) = 0
setsockopt(14, SOL_SOCKET, SO_KEEPALIVE, [0], 4) = 0
setsockopt(14, SOL_IP, IP_TOS, [8], 4) = 0
setsockopt(14, SOL_TCP, TCP_NODELAY, [1], 4) = 0
fcntl(14, F_GETFL) = 0x2 (flags O_RDWR)
fcntl(14, F_SETFL, O_RDWR) = 0
setsockopt(14, SOL_SOCKET, SO_KEEPALIVE, [0], 4) = 0
setsockopt(14, SOL_IP, IP_TOS, [8], 4) = 0
setsockopt(14, SOL_TCP, TCP_NODELAY, [1], 4) = 0
fcntl(14, F_GETFL) = 0x2 (flags O_RDWR)
fcntl(14, F_SETFL, O_RDWR) = 0
clone(child_stack=0x43111220,
flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID,
parent_tidptr=0x431119d0, tls=0x43111940, child_tidptr=0x431119d0) =
6299
clone(child_stack=0x43b12220,
flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID,
parent_tidptr=0x43b129d0, tls=0x43b12940, child_tidptr=0x43b129d0) =
6300
select(7, [6], NULL, NULL, NULL
Any other
On Thu, May 14, 2009 at 6:22 AM, Peter Boncz
Could you provide an strace (start the server with "strace" in front of the command)?
On May 14, 2009, at 3:03 PM, "Mag Gam"
wrote: Ok. I spoke to several people at Lustre and they say mmap() is implemented properly on Lustre and any application that uses mmap() should work.
Is there a way to simulate a mmap() call (I am not an expert programmer). I would like to see if this is a legemit bug on MonetDB or Lustre.
Would anyone like to help me with this?
TIA
On Wed, May 13, 2009 at 8:59 AM, Peter Boncz
wrote: I have never heard of Lustre and I am not aware of anyone using that with MonetDB.. A few years ago MonetDB still worked with (nfs) mmapped filed, but it was slow and their were consistency risks as dirtypages are not synced immediate over the network. For these reasons nobody used it. Then, it stopped working. We should clean up the crash and just make it fail or warn gracefully.
MonetDB does pretty scary stuff with mmap so without studying Lustre I cannot say wheter it could work.
On May 13, 2009, at 2:32 PM, "Mag Gam"
wrote: Thanks for the response Peter. This is on the Lustre filesystem. http://arch.lustre.org/images/9/95/Mmap_hld.pdf
I believe it supports it without any issues. Or could this be an issue?
On Wed, May 13, 2009 at 8:29 AM, Peter Boncz
wrote: Databases on NFS are not supported and lead to crashes on a mmaped dbfarm afaik.
On May 13, 2009, at 2:13 PM, "Mag Gam"
wrote: Yes, have to be on a network filesystem. Its pretty fast, I can get close to 500MB/sec consistently. Also, it has over 50TB of stable available :-)
This issue is getting very frustrating. :-(
On Wed, May 13, 2009 at 7:49 AM, Fabian Groffen
wrote: > On 13-05-2009 04:34:04 -0700, Mag Gam wrote: >> More information about the server: >> >> >> MonetDB server v5.10.4 (64-bit), based on kernel v1.28.4 (64-bit >> oids) >> Copyright (c) 1993-July 2008 CWI >> Copyright (c) August 2008-2009 MonetDB B.V., all rights reserved >> Visit http://monetdb.cwi.nl/ for further information >> >> >> Libraries: >> libpcre: 6.6 06-Feb-2006 (compiled with 6.6) >> openssl: OpenSSL 0.9.8b 04 May 2006 (compiled with OpenSSL >> 0.9.8b 04 May 2006) >> libxml2: 2.6.26 (compiled with 2.6.26) >> >> Compilation: gcc -O2 -std=c99 >> Linking : ld -IPA -m elf_x86_64 >> >> Also, >> >> ${prefix}/var/MonetDB5/dbfarm exists on a network filesystem. >> I am >> hoping thats ok (with flocking) > > Ohhh... Never tested, but that may be (a lot) slower... Do you > really > have to be on a network share? For speed it is regardless a good > idea > to have the storage local. mmapping on network shares might also > be a > bit less optimal. > > --- > --- > --- > --- > --- > --------------------------------------------------------------- > The NEW KODAK i700 Series Scanners deliver under ANY > circumstances! > Your > production scanning environment may not be a perfect world - but > thanks to > Kodak, there's a perfect scanner to get the job done! With the > NEW > KODAK i700 > Series Scanner you'll get full speed at 300 dpi even with all > image > processing features enabled. http://p.sf.net/sfu/kodak-com > _______________________________________________ > MonetDB-users mailing list > MonetDB-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-users > --- --- --- --- ------------------------------------------------------------------ The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your production scanning environment may not be a perfect world - but thanks to Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700 Series Scanner you'll get full speed at 300 dpi even with all image processing features enabled. http://p.sf.net/sfu/kodak-com _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
Any thoughts to this?
TIA
On 5/14/09, Mag Gam
strace on the process (mserver) just gives:
nanosleep({5, 0}, {5, 0}) = 0 nanosleep({5, 0}, {5, 0}) = 0 nanosleep({5, 0}, {5, 0}) = 0 nanosleep({5, 0}, {5, 0}) = 0 nanosleep({5, 0}, {5, 0}) = 0 nanosleep({5, 0}, {5, 0}) = 0
strace on merovrian give me, when I run mclient -l sql dbname:
read(16, "", 2097152) = 0 close(16) = 0 munmap(0x2aaaaaeca000, 2097152) = 0 close(14) = 0 stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=3519, ...}) = 0 write(1, "database \'DataBase\' already running s"..., 128) = 128 getpeername(12, {sa_family=AF_INET, sin_port=htons(41314), sin_addr=inet_addr("127.0.0.1")}, [16]) = 0 write(1, "proxying client 127.0.0.1:41314 "..., 89) = 89 write(12, "1\0", 2) = 2 write(12, "^mapi:merovingian:proxy\n", 24) = 24 open("/etc/hosts", O_RDONLY) = 14 fcntl(14, F_GETFD) = 0 fcntl(14, F_SETFD, FD_CLOEXEC) = 0 fstat(14, {st_mode=S_IFREG|0644, st_size=306, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aaaaaeca000 read(14, "127.0.0.1\t\tlocalhost\n141.128.90."..., 4096) = 306 close(14) = 0 munmap(0x2aaaaaeca000, 4096) = 0 socket(PF_INET, SOCK_STREAM, IPPROTO_TCP) = 14 connect(14, {sa_family=AF_INET, sin_port=htons(50001), sin_addr=inet_addr("141.128.90.133")}, 16) = 0 setsockopt(14, SOL_SOCKET, SO_KEEPALIVE, [0], 4) = 0 setsockopt(14, SOL_IP, IP_TOS, [8], 4) = 0 setsockopt(14, SOL_TCP, TCP_NODELAY, [1], 4) = 0 fcntl(14, F_GETFL) = 0x2 (flags O_RDWR) fcntl(14, F_SETFL, O_RDWR) = 0 setsockopt(14, SOL_SOCKET, SO_KEEPALIVE, [0], 4) = 0 setsockopt(14, SOL_IP, IP_TOS, [8], 4) = 0 setsockopt(14, SOL_TCP, TCP_NODELAY, [1], 4) = 0 fcntl(14, F_GETFL) = 0x2 (flags O_RDWR) fcntl(14, F_SETFL, O_RDWR) = 0 clone(child_stack=0x43111220, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x431119d0, tls=0x43111940, child_tidptr=0x431119d0) = 6299 clone(child_stack=0x43b12220, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x43b129d0, tls=0x43b12940, child_tidptr=0x43b129d0) = 6300 select(7, [6], NULL, NULL, NULL
Any other
On Thu, May 14, 2009 at 6:22 AM, Peter Boncz
wrote: Could you provide an strace (start the server with "strace" in front of the command)?
On May 14, 2009, at 3:03 PM, "Mag Gam"
wrote: Ok. I spoke to several people at Lustre and they say mmap() is implemented properly on Lustre and any application that uses mmap() should work.
Is there a way to simulate a mmap() call (I am not an expert programmer). I would like to see if this is a legemit bug on MonetDB or Lustre.
Would anyone like to help me with this?
TIA
On Wed, May 13, 2009 at 8:59 AM, Peter Boncz
wrote: I have never heard of Lustre and I am not aware of anyone using that with MonetDB.. A few years ago MonetDB still worked with (nfs) mmapped filed, but it was slow and their were consistency risks as dirtypages are not synced immediate over the network. For these reasons nobody used it. Then, it stopped working. We should clean up the crash and just make it fail or warn gracefully.
MonetDB does pretty scary stuff with mmap so without studying Lustre I cannot say wheter it could work.
On May 13, 2009, at 2:32 PM, "Mag Gam"
wrote: Thanks for the response Peter. This is on the Lustre filesystem. http://arch.lustre.org/images/9/95/Mmap_hld.pdf
I believe it supports it without any issues. Or could this be an issue?
On Wed, May 13, 2009 at 8:29 AM, Peter Boncz
wrote: Databases on NFS are not supported and lead to crashes on a mmaped dbfarm afaik.
On May 13, 2009, at 2:13 PM, "Mag Gam"
wrote: > Yes, have to be on a network filesystem. Its pretty fast, I can > get > close to 500MB/sec consistently. Also, it has over 50TB of stable > available :-) > > This issue is getting very frustrating. :-( > > > On Wed, May 13, 2009 at 7:49 AM, Fabian Groffen >
wrote: >> On 13-05-2009 04:34:04 -0700, Mag Gam wrote: >>> More information about the server: >>> >>> >>> MonetDB server v5.10.4 (64-bit), based on kernel v1.28.4 (64-bit >>> oids) >>> Copyright (c) 1993-July 2008 CWI >>> Copyright (c) August 2008-2009 MonetDB B.V., all rights reserved >>> Visit http://monetdb.cwi.nl/ for further information >>> >>> >>> Libraries: >>> libpcre: 6.6 06-Feb-2006 (compiled with 6.6) >>> openssl: OpenSSL 0.9.8b 04 May 2006 (compiled with OpenSSL >>> 0.9.8b 04 May 2006) >>> libxml2: 2.6.26 (compiled with 2.6.26) >>> >>> Compilation: gcc -O2 -std=c99 >>> Linking : ld -IPA -m elf_x86_64 >>> >>> Also, >>> >>> ${prefix}/var/MonetDB5/dbfarm exists on a network filesystem. >>> I am >>> hoping thats ok (with flocking) >> >> Ohhh... Never tested, but that may be (a lot) slower... Do you >> really >> have to be on a network share? For speed it is regardless a good >> idea >> to have the storage local. mmapping on network shares might also >> be a >> bit less optimal. >> >> --- >> --- >> --- >> --- >> --- >> --------------------------------------------------------------- >> The NEW KODAK i700 Series Scanners deliver under ANY >> circumstances! >> Your >> production scanning environment may not be a perfect world - but >> thanks to >> Kodak, there's a perfect scanner to get the job done! With the >> NEW >> KODAK i700 >> Series Scanner you'll get full speed at 300 dpi even with all >> image >> processing features enabled. http://p.sf.net/sfu/kodak-com >> _______________________________________________ >> MonetDB-users mailing list >> MonetDB-users@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/monetdb-users >> > > --- > --- > --- > --- > ------------------------------------------------------------------ > The NEW KODAK i700 Series Scanners deliver under ANY > circumstances! > Your > production scanning environment may not be a perfect world - but > thanks to > Kodak, there's a perfect scanner to get the job done! With the NEW > KODAK i700 > Series Scanner you'll get full speed at 300 dpi even with all > image > processing features enabled. http://p.sf.net/sfu/kodak-com > _______________________________________________ > MonetDB-users mailing list > MonetDB-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-users
Excuse me. To trace thus, start mserver (with strace) by itself,
without merovigian.. The nanosleeps are past the crucial point.
On May 28, 2009, at 11:46 PM, "Mag Gam"
Any thoughts to this?
TIA
On 5/14/09, Mag Gam
wrote: strace on the process (mserver) just gives:
nanosleep({5, 0}, {5, 0}) = 0 nanosleep({5, 0}, {5, 0}) = 0 nanosleep({5, 0}, {5, 0}) = 0 nanosleep({5, 0}, {5, 0}) = 0 nanosleep({5, 0}, {5, 0}) = 0 nanosleep({5, 0}, {5, 0}) = 0
strace on merovrian give me, when I run mclient -l sql dbname:
read(16, "", 2097152) = 0 close(16) = 0 munmap(0x2aaaaaeca000, 2097152) = 0 close(14) = 0 stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=3519, ...}) = 0 write(1, "database \'DataBase\' already running s"..., 128) = 128 getpeername(12, {sa_family=AF_INET, sin_port=htons(41314), sin_addr=inet_addr("127.0.0.1")}, [16]) = 0 write(1, "proxying client 127.0.0.1:41314 "..., 89) = 89 write(12, "1\0", 2) = 2 write(12, "^mapi:merovingian:proxy\n", 24) = 24 open("/etc/hosts", O_RDONLY) = 14 fcntl(14, F_GETFD) = 0 fcntl(14, F_SETFD, FD_CLOEXEC) = 0 fstat(14, {st_mode=S_IFREG|0644, st_size=306, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aaaaaeca000 read(14, "127.0.0.1\t\tlocalhost\n141.128.90."..., 4096) = 306 close(14) = 0 munmap(0x2aaaaaeca000, 4096) = 0 socket(PF_INET, SOCK_STREAM, IPPROTO_TCP) = 14 connect(14, {sa_family=AF_INET, sin_port=htons(50001), sin_addr=inet_addr("141.128.90.133")}, 16) = 0 setsockopt(14, SOL_SOCKET, SO_KEEPALIVE, [0], 4) = 0 setsockopt(14, SOL_IP, IP_TOS, [8], 4) = 0 setsockopt(14, SOL_TCP, TCP_NODELAY, [1], 4) = 0 fcntl(14, F_GETFL) = 0x2 (flags O_RDWR) fcntl(14, F_SETFL, O_RDWR) = 0 setsockopt(14, SOL_SOCKET, SO_KEEPALIVE, [0], 4) = 0 setsockopt(14, SOL_IP, IP_TOS, [8], 4) = 0 setsockopt(14, SOL_TCP, TCP_NODELAY, [1], 4) = 0 fcntl(14, F_GETFL) = 0x2 (flags O_RDWR) fcntl(14, F_SETFL, O_RDWR) = 0 clone(child_stack=0x43111220, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD| CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x431119d0, tls=0x43111940, child_tidptr=0x431119d0) = 6299 clone(child_stack=0x43b12220, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD| CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x43b129d0, tls=0x43b12940, child_tidptr=0x43b129d0) = 6300 select(7, [6], NULL, NULL, NULL
Any other
On Thu, May 14, 2009 at 6:22 AM, Peter Boncz
wrote: Could you provide an strace (start the server with "strace" in front of the command)?
On May 14, 2009, at 3:03 PM, "Mag Gam"
wrote: Ok. I spoke to several people at Lustre and they say mmap() is implemented properly on Lustre and any application that uses mmap() should work.
Is there a way to simulate a mmap() call (I am not an expert programmer). I would like to see if this is a legemit bug on MonetDB or Lustre.
Would anyone like to help me with this?
TIA
On Wed, May 13, 2009 at 8:59 AM, Peter Boncz
wrote: I have never heard of Lustre and I am not aware of anyone using that with MonetDB.. A few years ago MonetDB still worked with (nfs) mmapped filed, but it was slow and their were consistency risks as dirtypages are not synced immediate over the network. For these reasons nobody used it. Then, it stopped working. We should clean up the crash and just make it fail or warn gracefully.
MonetDB does pretty scary stuff with mmap so without studying Lustre I cannot say wheter it could work.
On May 13, 2009, at 2:32 PM, "Mag Gam"
wrote: Thanks for the response Peter. This is on the Lustre filesystem. http://arch.lustre.org/images/9/95/Mmap_hld.pdf
I believe it supports it without any issues. Or could this be an issue?
On Wed, May 13, 2009 at 8:29 AM, Peter Boncz
wrote: > Databases on NFS are not supported and lead to crashes on a > mmaped > dbfarm afaik. > > On May 13, 2009, at 2:13 PM, "Mag Gam" > wrote: > >> Yes, have to be on a network filesystem. Its pretty fast, I can >> get >> close to 500MB/sec consistently. Also, it has over 50TB of >> stable >> available :-) >> >> This issue is getting very frustrating. :-( >> >> >> On Wed, May 13, 2009 at 7:49 AM, Fabian Groffen >> wrote: >>> On 13-05-2009 04:34:04 -0700, Mag Gam wrote: >>>> More information about the server: >>>> >>>> >>>> MonetDB server v5.10.4 (64-bit), based on kernel v1.28.4 >>>> (64-bit >>>> oids) >>>> Copyright (c) 1993-July 2008 CWI >>>> Copyright (c) August 2008-2009 MonetDB B.V., all rights >>>> reserved >>>> Visit http://monetdb.cwi.nl/ for further information >>>> >>>> >>>> Libraries: >>>> libpcre: 6.6 06-Feb-2006 (compiled with 6.6) >>>> openssl: OpenSSL 0.9.8b 04 May 2006 (compiled with OpenSSL >>>> 0.9.8b 04 May 2006) >>>> libxml2: 2.6.26 (compiled with 2.6.26) >>>> >>>> Compilation: gcc -O2 -std=c99 >>>> Linking : ld -IPA -m elf_x86_64 >>>> >>>> Also, >>>> >>>> ${prefix}/var/MonetDB5/dbfarm exists on a network filesystem. >>>> I am >>>> hoping thats ok (with flocking) >>> >>> Ohhh... Never tested, but that may be (a lot) slower... Do >>> you >>> really >>> have to be on a network share? For speed it is regardless a >>> good >>> idea >>> to have the storage local. mmapping on network shares might >>> also >>> be a >>> bit less optimal. >>> >>> --- >>> --- >>> --- >>> --- >>> --- >>> --- >>> ------------------------------------------------------------ >>> The NEW KODAK i700 Series Scanners deliver under ANY >>> circumstances! >>> Your >>> production scanning environment may not be a perfect world - >>> but >>> thanks to >>> Kodak, there's a perfect scanner to get the job done! With the >>> NEW >>> KODAK i700 >>> Series Scanner you'll get full speed at 300 dpi even with all >>> image >>> processing features enabled. http://p.sf.net/sfu/kodak-com >>> _______________________________________________ >>> MonetDB-users mailing list >>> MonetDB-users@lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/monetdb-users >>> >> >> --- >> --- >> --- >> --- >> --- >> --------------------------------------------------------------- >> The NEW KODAK i700 Series Scanners deliver under ANY >> circumstances! >> Your >> production scanning environment may not be a perfect world - >> but >> thanks to >> Kodak, there's a perfect scanner to get the job done! With >> the NEW >> KODAK i700 >> Series Scanner you'll get full speed at 300 dpi even with all >> image >> processing features enabled. http://p.sf.net/sfu/kodak-com >> _______________________________________________ >> MonetDB-users mailing list >> MonetDB-users@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/monetdb-users > --- --- --- --------------------------------------------------------------------- Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT is a gathering of tech-side developers & brand creativity professionals. Meet the minds behind Google Creative Lab, Visual Complexity, Processing, & iPhoneDevCamp as they present alongside digital heavyweights like Barbarian Group, R/GA, & Big Spaceship. http://p.sf.net/sfu/creativitycat-com _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
participants (3)
-
Fabian Groffen
-
Mag Gam
-
Peter Boncz