On Mon, Mar 17, 2008 at 12:15:11AM +0100, Michael Schmidt wrote:
Hi,
I am trying to run the following SQL query on the lates nightly build:
SELECT T1.subject AS inproc, T2.object AS author, T3.object AS booktitle, T4.object AS title, T5.object AS proc, T6.object AS ee, T7.object AS page, T8.object AS URL, T9.object AS yr, AB.object AS abstract FROM Triples T1 JOIN Triples T2 ON T1.subject=T2.subject JOIN Triples T3 ON T1.subject=T3.subject JOIN Triples T4 ON T1.subject=T4.subject JOIN Triples T5 ON T1.subject=T5.subject JOIN Triples T6 ON T1.subject=T6.subject JOIN Triples T7 ON T1.subject=T7.subject JOIN Triples T8 ON T1.subject=T8.subject JOIN Triples T9 ON T1.subject=T9.subject LEFT JOIN ( SELECT * FROM Triples T10 WHERE T10.property='bench:abstract' ) AB ON T1.subject=AB.subject WHERE T1.property='rdf:type' AND T2.property='dc:creator' AND T3.property='bench:booktitle' AND T4.property='dc:title' AND T5.property='dcterms:partOf' AND T6.property='rdfs:seeAlso' AND T7.property='swrc:pages' AND T8.property='foaf:homepage' AND T9.property='dcterms:issued' AND T1.object='bench:Inproceedings' ORDER BY T9.object;
on a single table "Triples(subject VARCHAR(50),predicate VARCHAR(20),object VARCHAR(20000))" with only 1000 entries. This query results in the following error message:
##GDKmmap(702808064) fail => BBPtrim(enter) usage[mem=101947960,vm=1505614392] # #mallinfo.arena = 11571200 ##mallinfo.ordblks = 1177 ##mallinfo.smblks = 102 ##mallinfo.hblkhd = 94076928 ##mallinfo.hblks = 12 ##mallinfo.usmblks = 0 ##mallinfo.fsmblks = 4080 ##mallinfo.uordblks = 9619432 ##mallinfo.fordblks = 1951768 ##BBPTRIM_ENTER: memsize=101947960,vmsize=1505614392 ##BBPTRIM: memtarget=0 vmtarget=1073741824 ##TRIMSCAN: mem=0 vm=1, start=1, limit=1 ##TRIMSCAN: 63756* 0=tmp_1462 (#0) ##TRIMSCAN: 63887* 1=tmp_1472 (#0) ##TRIMSCAN: 64223* 2=tmp_1473 (#0) ##TRIMSCAN: 64286* 3=tmp_1477 (#0) ##TRIMSCAN: 64301* 4=tmp_1500 (#0) ##TRIMSCAN: 64362* 5=tmp_1501 (#0) ##TRIMSCAN: 64438* 6=tmp_1502 (#0) ##TRIMSCAN: 64447* 7=tmp_1505 (#0) [...] // many many more entries of this form ##TRIMSCAN: 63994 235=tmp_2275 (#455647) ##TRIMSCAN: 63998 236=tmp_2304 (#3980331) ##TRIMSCAN: 64002 237=tmp_2305 (#3980331) ##TRIMSCAN: end at 1 (size=1234) ##TRIMSELECT: dirty = 0 ##TRIMSELECT: candidate=tmp_1462 BAT*=0x848fc28 ## (cnt=0, mode=4096, refs=0, wait=0, parent=0,0, lastused=51285,63756,-2147419892) ##TRIMSELECT: keep tmp_1462 [228,0] bytes [228,0] dirty target(mem=0 vm=1073741824) ##TRIMSELECT: candidate=tmp_1472 BAT*=0x84964f8 ## (cnt=0, mode=4096, refs=0, wait=0, parent=0,0, lastused=51294,63887,-2147419761) ##TRIMSELECT: keep tmp_1472 [228,0] bytes [228,0] dirty target(mem=0 vm=1073741824) ##TRIMSELECT: candidate=tmp_1473 BAT*=0x8496b18 ## (cnt=0, mode=4096, refs=0, wait=0, parent=0,0, lastused=51313,64223,-2147419425) ##TRIMSELECT: keep tmp_1473 [228,0] bytes [0,0] dirty target(mem=0 vm=1073741824) ##TRIMSELECT: candidate=tmp_1477 BAT*=0x84a6770 ## (cnt=0, mode=4096, refs=0, wait=0, parent=0,0, lastused=51321,64286,-2147419362) ##TRIMSELECT: keep tmp_1477 [228,0] bytes [0,0] dirty target(mem=0 vm=1073741824) ##TRIMSELECT: candidate=tmp_1500 BAT*=0x84a6d20 ## (cnt=0, mode=4096, refs=0, wait=0, parent=0,0, lastused=51323,64301,-2147419347) ##TRIMSELECT: keep tmp_1500 [228,0] bytes [0,0] dirty target(mem=0 vm=1073741824) ##TRIMSELECT: candidate=tmp_1501 BAT*=0x84a7318 ## (cnt=0, mode=4096, refs=0, wait=0, parent=0,0, lastused=51325,64362,-2147419286) ##TRIMSELECT: keep tmp_1501 [228,0] bytes [0,0] dirty target(mem=0 vm=1073741824) ##TRIMSELECT: candidate=tmp_1502 BAT*=0x84a7920 [...] // many many more entries of this form ##TRIMSELECT: candidate=tmp_2234 BAT*=0x871d5b0 ## (cnt=0, mode=1024, refs=0, wait=0, parent=0,0, lastused=63975,63975,63975) ##TRIMSELECT: delete tmp_2234 from trimlist (does not match trim needs) ##TRIMSELECT: candidate=tmp_2235 BAT*=0x871e7e8 ## (cnt=0, mode=1024, refs=0, wait=0, parent=0,0, lastused=63976,63976,63976) ##TRIMSELECT: delete tmp_2235 from trimlist (does not match trim needs) ##TRIMSELECT: candidate=tmp_2236 BAT*=0x871fa20 ## (cnt=0, mode=1024, refs=0, wait=0, parent=0,0, lastused=63977,63977,63977) ##TRIMSELECT: delete tmp_2236 from trimlist (does not match trim needs) ##TRIMSELECT: end ##BBPTRIM: no more unload candidates! ##BBPTRIM_EXIT: memsize=11583488,vmsize=1505614392 ##GDKmmap(702808064) fail => BBPtrim(ready) usage[mem=101947960,vm=1505614392] # #mallinfo.arena = 11571200 ##mallinfo.ordblks = 1177 ##mallinfo.smblks = 102 ##mallinfo.hblkhd = 94076928 ##mallinfo.hblks = 12 ##mallinfo.usmblks = 0 ##mallinfo.fsmblks = 4080 ##mallinfo.uordblks = 9619432
Seems like a memory allocation problem to me. Any ideas what the problem could be?
It seems we failed to push the selections down, ie many of the joins are done first. This easily explodes, and therefor no memory is left. Maybe we a bit of 'sql' rewriting you may work around the problem. Anyway could you file this as a bug via our sourceforge bug reporting system? Niels
Best Regards, Michael
------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
-- Niels Nes, Centre for Mathematics and Computer Science (CWI) Kruislaan 413, 1098 SJ Amsterdam, The Netherlands room C0.02, phone ++31 20 592-4098, fax ++31 20 592-4312 url: http://www.cwi.nl/~niels e-mail: Niels.Nes@cwi.nl