[Monetdb-developers] Unable to shred 1gb xml file (OS: Not enough space)

Hi, I tried MonetDB/XQuery to shred a 1gb xml file, but it failed. Following is the error message: MonetDB>shred_doc("D:xbench/output/DC1000catalog.xml", "DC1000catalog.xml"); !ERROR: MT_mmap: MapViewOfFile(6b4, 2, 0, 0, 442499072, 0) failed !OS: Not enough space !GDKmmap(442499072) fail => BBPtrim(enter) usage[mem=101297568,vm=794361856] I use windows XP Professional version 2002 service pack 2, Pentium 4 CPU, 2.40GHz, 512 MB of RAM. Previously I tried to shred 100mb xml file, and it took 18.532 sec. For this 1 gb file, I left it overnight so I don't know how long it takes till the error message occurs. Can please help me to solve this problem? Regards, Klarinda Below is the complete error message: MonetDB>shred_doc("D:xbench/output/DC1000catalog.xml", "DC1000catalog.xml"); !ERROR: MT_mmap: MapViewOfFile(6b4, 2, 0, 0, 442499072, 0) failed !OS: Not enough space !GDKmmap(442499072) fail => BBPtrim(enter) usage[mem=101297568,vm=794361856] # !mallinfo.arena = 15613828 !mallinfo.ordblks = 46134 !mallinfo.smblks = 15492 !mallinfo.hblkhd = 0 !mallinfo.hblks = 0 !mallinfo.usmblks = 13718408 !mallinfo.fsmblks = 899720 !mallinfo.uordblks = 950740 !mallinfo.fordblks = 44960 #BBPTRIM_ENTER: memsize=101297568,vmsize=794361856 #BBPTRIM: memtarget=0 vmtarget=1073741824 #TRIMSCAN: mem=0 vm=1, start=1, limit=1 #TRIMSCAN: 145030 0=tmp_35 (#0) #TRIMSCAN: 145059 1=tmp_36 (#0) #TRIMSCAN: 145088 2=tmp_37 (#0) #TRIMSCAN: 145146 3=tmp_41 (#0) #TRIMSCAN: 149075 4=doc_query (#0) #TRIMSCAN: 149092 5=doc_sema (#0) #TRIMSCAN: 155215 6=tmp_374 (#0) #TRIMSCAN: 155218 7=prop_pre_39 (#0) #TRIMSCAN: 157895 8=tmp_533 (#0) #TRIMSCAN: 157898 9=prop_pre_310 (#0) #TRIMSCAN: end at 1 (size=628) #TRIMSELECT: dirty = 0 #TRIMSELECT: candidate=tmp_35 BAT*=03D6A230 # (cnt=0, mode=1024, refs=0, wait=0, parent=0, lastused=145030,145030 ,145030) #TRIMSELECT: keep tmp_35 [224,0] bytes [224,0] dirty target(mem=0 vm=1073741824) #TRIMSELECT: candidate=tmp_36 BAT*=03D66E60 # (cnt=0, mode=1024, refs=0, wait=0, parent=0, lastused=145059,145059 ,145059) #TRIMSELECT: keep tmp_36 [224,0] bytes [224,0] dirty target(mem=0 vm=1073741824) #TRIMSELECT: candidate=tmp_37 BAT*=058080B0 # (cnt=0, mode=1024, refs=0, wait=0, parent=0, lastused=145088,145088 ,145088) #TRIMSELECT: keep tmp_37 [224,0] bytes [224,0] dirty target(mem=0 vm=1073741824) #TRIMSELECT: candidate=tmp_41 BAT*=058066F0 # (cnt=0, mode=1024, refs=0, wait=0, parent=0, lastused=145146,145146 ,145146) #TRIMSELECT: keep tmp_41 [224,0] bytes [224,0] dirty target(mem=0 vm=1073741824) #TRIMSELECT: candidate=doc_query BAT*=057F9D10 # (cnt=0, mode=1024, refs=0, wait=0, parent=0, lastused=149075,149075 ,149075) #TRIMSELECT: keep doc_query [224,0] bytes [224,0] dirty target(mem=0 vm=10737418 24) #TRIMSELECT: candidate=doc_sema BAT*=05845370 # (cnt=0, mode=1024, refs=0, wait=0, parent=0, lastused=149092,149092 ,149092) #TRIMSELECT: keep doc_sema [224,0] bytes [224,0] dirty target(mem=0 vm=107374182 4) #TRIMSELECT: candidate=tmp_374 BAT*=03D722F0 # (cnt=0, mode=4096, refs=0, wait=0, parent=0, lastused=155215,155215 ,155215) #TRIMSELECT: keep tmp_374 [224,0] bytes [0,0] dirty target(mem=0 vm=1073741824) #TRIMSELECT: candidate=prop_pre_39 BAT*=05845CD0 # (cnt=0, mode=4096, refs=0, wait=0, parent=0, lastused=155218,155218 ,155218) #TRIMSELECT: keep prop_pre_39 [224,0] bytes [0,0] dirty target(mem=0 vm=10737418 24) #TRIMSELECT: candidate=tmp_533 BAT*=0583C598 # (cnt=0, mode=4096, refs=0, wait=0, parent=0, lastused=157895,157895 ,157895) #TRIMSELECT: keep tmp_533 [224,0] bytes [0,0] dirty target(mem=0 vm=1073741824) #TRIMSELECT: candidate=prop_pre_310 BAT*=057F7E70 # (cnt=0, mode=4096, refs=0, wait=0, parent=0, lastused=157898,157898 ,157898) #TRIMSELECT: keep prop_pre_310 [224,0] bytes [0,0] dirty target(mem=0 vm=1073741 824) #TRIMSELECT: end #TRIMSELECT: dirty = 1 #TRIMSELECT: candidate=tmp_35 BAT*=03D6A230 # (cnt=0, mode=1024, refs=0, wait=0, parent=0, lastused=145030,145030 ,145030) #TRIMSELECT: delete tmp_35 from trimlist (does not match trim needs) #TRIMSELECT: candidate=tmp_36 BAT*=03D66E60 # (cnt=0, mode=1024, refs=0, wait=0, parent=0, lastused=145059,145059 ,145059) #TRIMSELECT: delete tmp_36 from trimlist (does not match trim needs) #TRIMSELECT: candidate=tmp_37 BAT*=058080B0 # (cnt=0, mode=1024, refs=0, wait=0, parent=0, lastused=145088,145088 ,145088) #TRIMSELECT: delete tmp_37 from trimlist (does not match trim needs) #TRIMSELECT: candidate=tmp_41 BAT*=058066F0 # (cnt=0, mode=1024, refs=0, wait=0, parent=0, lastused=145146,145146 ,145146) #TRIMSELECT: delete tmp_41 from trimlist (does not match trim needs) #TRIMSELECT: candidate=doc_query BAT*=057F9D10 # (cnt=0, mode=1024, refs=0, wait=0, parent=0, lastused=149075,149075 ,149075) #TRIMSELECT: delete doc_query from trimlist (does not match trim needs) #TRIMSELECT: candidate=doc_sema BAT*=05845370 # (cnt=0, mode=1024, refs=0, wait=0, parent=0, lastused=149092,149092 ,149092) #TRIMSELECT: delete doc_sema from trimlist (does not match trim needs) #TRIMSELECT: candidate=tmp_374 BAT*=03D722F0 # (cnt=0, mode=4096, refs=0, wait=0, parent=0, lastused=155215,155215 ,155215) #TRIMSELECT: delete tmp_374 from trimlist (does not match trim needs) #TRIMSELECT: candidate=prop_pre_39 BAT*=05845CD0 # (cnt=0, mode=4096, refs=0, wait=0, parent=0, lastused=155218,155218 ,155218) #TRIMSELECT: delete prop_pre_39 from trimlist (does not match trim needs) #TRIMSELECT: candidate=tmp_533 BAT*=0583C598 # (cnt=0, mode=4096, refs=0, wait=0, parent=0, lastused=157895,157895 ,157895) #TRIMSELECT: delete tmp_533 from trimlist (does not match trim needs) #TRIMSELECT: candidate=prop_pre_310 BAT*=057F7E70 # (cnt=0, mode=4096, refs=0, wait=0, parent=0, lastused=157898,157898 ,157898) #TRIMSELECT: delete prop_pre_310 from trimlist (does not match trim needs) #TRIMSELECT: end #BBPTRIM: no more unload candidates! #BBPTRIM_EXIT: memsize=95140356,vmsize=794361856 !GDKmmap(442499072) fail => BBPtrim(ready) usage[mem=101297568,vm=794361856] # !mallinfo.arena = 15613828 !mallinfo.ordblks = 46134 !mallinfo.smblks = 15492 !mallinfo.hblkhd = 0 !mallinfo.hblks = 0 !mallinfo.usmblks = 13718408 !mallinfo.fsmblks = 899720 !mallinfo.uordblks = 950740 !mallinfo.fordblks = 44960 !ERROR: MT_mmap: MapViewOfFile(6b0, 2, 0, 0, 442499072, 0) failed !OS: Not enough space !ERROR: GDKload: cannot mmap(): name=05\552, ext=theap.priv !OS: Not enough space !ERROR: GDKload failed: name=05\552, ext=theap.priv !ERROR: shredder.mx:append_str2bat: APPEND-STR[PROP_TEXT](final foxes since the silent, quick realms should breach never sheaves--ruthless, daring waters beneat h the close asymptotes c), BUNappend fails !ERROR: CMDshred2bats: operation failed. MonetDB>