Hi all,
I've been testing MonetDB and I've been getting strange memory
allocation errors while running large 'group by' queries. From a dataset
of around 60GB, I'm running 'group by' queries against it to test for
speed. Queries of the sort:
select keyword, count(*) as c
from raw_data
group by keyword
order by c desc
limit 100;
What's happening is that it will trundle along ok for a while, generally
consuming all the available memory (98%) on 3GB box. But after a while I
get these messages on the server:
# Listening for connection requests on mapi:monetdb://127.0.0.1:50000/
# MonetDB/SQL module v2.28.4 loaded
>#GDKmmap(10412883968) fails, try to free up space [memory in
use=72431192,virtual memory in use=71134150656]
#GDKmmap(10412883968) result [mem=71387384,vm=71134150656]
#GDKvmalloc(10412883984) fails, try to free up space [memory in
use=71386904,virtual memory in use=71134150656]
#GDKvmalloc(10412883984) result [mem=71130480,vm=71134150656]
!ERROR: HEAPalloc: Insufficient space for HEAP of 10412883968 bytes.
Also while checking top it will reports a much large RES size than
physical memory. E.g. (250% for %MEM).
I haven't changed any out of the box settings. Is there a minimum memory
size that MonetDB requires to play nice? Is there anywhere I can go to
limit the amount of memory MonetDB requires?
The version I'm using is:
mserver5 --version
MonetDB server v5.10.4 (64-bit), based on kernel v1.28.4 (64-bit oids)
Copyright (c) 1993-July 2008 CWI
Copyright (c) August 2008-2009 MonetDB B.V., all rights reserved
Visit http://monetdb.cwi.nl/ for further information
Configured for prefix: /usr
Libraries:
libpcre: 6.6 06-Feb-2006 (compiled with 6.6)
openssl: OpenSSL 0.9.8e-rhel5 01 Jul 2008 (compiled with OpenSSL
0.9.8e-fips-rhel5 01 Jul 2008)
libxml2: 2.6.26 (compiled with 2.6.26)
Compiled by: mockbuild(a)surya.karan.org
Compilation: gcc -O2 -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions
-fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -std=c99
-O6 -fomit-frame-pointer -finline-functions
-falign-loops=4 -falign-jumps=4 -falign-functions=4
-fexpensive-optimizations -funroll-loops
-frerun-cse-after-loop -frerun-loop-opt -ftree-vectorize
Linking : ld -IPA -m elf_x86_64
Cheers
Hi all,
I'm having problems with MonetDB5 while performing LIKE queries against
it. Normal queries work fine, but anything with a LIKE in it gets this
response:
sql> select * from raw_data where keyword like 'apple%' limit 1;
!MALException:batcalc.str:can not create bat
0 tuples
Any ideas?
This is the version of mserver5 I'm using:
MonetDB server v5.10.4 (64-bit), based on kernel v1.28.4 (64-bit oids)
Copyright (c) 1993-July 2008 CWI
Copyright (c) August 2008-2009 MonetDB B.V., all rights reserved
Visit http://monetdb.cwi.nl/ for further information
Configured for prefix: /usr
Libraries:
libpcre: 6.6 06-Feb-2006 (compiled with 6.6)
openssl: OpenSSL 0.9.8e-rhel5 01 Jul 2008 (compiled with OpenSSL
0.9.8e-fips-rhel5 01 Jul 2008)
libxml2: 2.6.26 (compiled with 2.6.26)
Compiled by: mockbuild(a)surya.karan.org
Compilation: gcc -O2 -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions
-fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -std=c99
-O6 -fomit-frame-pointer -finline-functions
-falign-loops=4 -falign-jumps=4 -falign-functions=4
-fexpensive-optimizations -funroll-loops
-frerun-cse-after-loop -frerun-loop-opt -ftree-vectorize
Linking : ld -IPA -m elf_x86_64
Cheers
Hi,
I am trying to build the new May2009 release from CVS on windows.
Unlike in previous releases, I get this error when compiling MonetDB5:
.\..\..\..\src\mal\mal_autorize.mx(97) : fatal error C1083: Cannot open
include file: 'openssl/md5.h': No such file or directory
There doesn't seem to be an 'openssl' directory anywhere.
Any ideas?
--
View this message in context: http://www.nabble.com/May2009-branch---missing-openssl-md5.h-tp23850193p238…
Sent from the monetdb-users mailing list archive at Nabble.com.
Hi,
I am using the MAPI programming interface to communicate with MonetDB on
windows.
I am using mapi_connect, mapi_query, mapi_disconnect and mapi_destroy as is
shown in the examples.
However, after querying for a long time, I eventually get an 'Out of client
slots' exception.
Moreover, it seems that ports aren't closed properly - If I run 'netstat' in
the windows command prompt to see all the open ports, I see a huge list of
open 50000 ports, which are used by MonetDB. These ports are marked with a
'TIME_WAIT' state, which I think means that they're inactive but not
released.
Could it be that I'm not using the MAPI interface properly? Or could it be a
bug?
I am running a compiled version of the Feb2009SP2 branch.
Thanks.
--
View this message in context: http://www.nabble.com/%27Out-of-client-slots%27-with-MAPI-tp23849981p238499…
Sent from the monetdb-users mailing list archive at Nabble.com.
