Thanks for the quick feedback, Stefan,
In case of doubt, we recomment to use a 64-bit version of MonetDB (on a 64-bit system); cf., http://monetdb.cwi.nl/XQuery/Documentation/Scalability.html
On this page, I found the information that "The default 64-bits MonetDB/XQuery binaries are built with 32-bits object identifiers (OIDs)". I noticed that the "--disable-oid32" isn't recognized by monetdb-install.sh; but this is probably negligible, as my Mserver instance shows me the following information: # MonetDB Server v4.38.5 # based on GDK v1.38.5 # release Jun2010-SP2 # Copyright (c) 1993-July 2008, CWI. All rights reserved. # Copyright (c) August 2008-2010, MonetDB B.V.. All rights reserved. # Compiled for x86_64-unknown-linux-gnu/64bit with 64bit OIDs; dynamically linked. …so I guess 64bit is now default? Next, I've now successfully created an 1GB XMark instance on a 64bit Linux machine with 32GB RAM; but when I try to shred an 11GB instance, I get the following output:
mclient -tlx pfadddoc.xq #GDKmmap(1336803328) fails, try to free up space [memory in use=33105304,virtual memory in use=36277911552] #GDKmmap(1336803328) result [mem=32625976,vm=35689922560] #GDKmmap: recovery ok. Continuing.. #GDKmmap(1336803328) fails, try to free up space [memory in use=32626608,virtual memory in use=36971282432] #GDKmmap(1336803328) result [mem=23481048,vm=36963221504] MAPI = monetdb@localhost:50000 QUERY = pf:add-doc("11gb.xml", "xmark") ERROR = !ERROR: HEAPalloc: Insufficient space for HEAP of 1336803328 bytes. !ERROR: CMDleftjoin: operation failed. !ERROR: interpret_params: reverse(param 1): evaluation error. Timer 720853.104 msec
Do you have an idea how to tweak MonetDB, or my system, to parse documents of that size? Currently, no other processes are running on the machine, and space on hard disk should be enough as well. Thanks again, Christian