On Mon, Jun 08, 2009 at 05:53:42PM +0000, Martin Kersten wrote:

Update of /cvsroot/monetdb/MonetDB5/src/optimizer In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv7310

Modified Files: Tag: May2009 opt_joinpath.mx Log Message: Experimental stable code to deal with too large intermediate joins. It marks them as 'postpone' until at least one other join in the path has been executed. Multiple variations can be envisioned.

U opt_joinpath.mx Index: opt_joinpath.mx =================================================================== RCS file: /cvsroot/monetdb/MonetDB5/src/optimizer/opt_joinpath.mx,v retrieving revision 1.47.2.1 retrieving revision 1.47.2.2 diff -u -d -r1.47.2.1 -r1.47.2.2 --- opt_joinpath.mx 8 Jun 2009 10:56:25 -0000 1.47.2.1 +++ opt_joinpath.mx 8 Jun 2009 17:53:40 -0000 1.47.2.2 @@ -408,7 +408,9 @@ { BAT *b; BUN estimate, e; - int i, j; + int i, j, k; + int *postpone= (int*) GDKzalloc(sizeof(int) *top); + int postponed=0;

/* solve the join by pairing the smallest first */ while (top > 2) { @@ -426,7 +428,7 @@ stream_printf(cntxt->fdout,"#joinPath estimate join(%d,%d) %d\n", joins[i]->batCacheid, joins[i+1]->batCacheid,(int)e); #endif - if (e < estimate) { + if (e < estimate && ( !(postpone[i] && postpone[i+1]) || postponed==top-1)) { estimate = e; j = i; } @@ -434,10 +436,28 @@ @- BEWARE. you may not use a size estimation, because it may fire a BATproperty check in a few cases. +In case a join fails, we may try another order first before +abandoning the task. @c b = BATjoin(joins[j], joins[j + 1], BUN_NONE); - if (b==NULL) - return NULL; /* something seriously went wrong */ + if (b==NULL){ + if ( postpone[j]){ ^^^^^^^^^^^ shouldn't this be (e.g.) postpone[j] && postpone[j+1] i.e., bailout (only) if the exact join has failed before, not if an other join with the same (i.e., join(j-1,j)) has failed before?

... just a spontaneous idea ... Stefan

+ GDKfree(postpone); + return NULL; + } + /* clear the GDKerrors */ + if( cntxt->errbuf ) + cntxt->errbuf[0]=0; + postpone[j] = TRUE; + postpone[j+1] = TRUE; + postponed++; + continue; + } else { + /* reset the postponed joins */ + for( k=0; kfdout, "#joinPath %d:= join(%d,%d)\n", b->batCacheid, joins[j]->batCacheid, joins[j + 1]->batCacheid); @@ -450,6 +470,7 @@ for (i = j + 1; i < top; i++) joins[i] = joins[i + 1]; } + GDKfree(postpone); #ifdef DEBUG_OPT_JOINPATH stream_printf(cntxt->fdout, "#joinPath final join %d and %d\n", joins[0]->batCacheid, joins[1]->batCacheid); #endif

------------------------------------------------------------------------------ Crystal Reports - New Free Runtime and 30 Day Trial Check out the new simplified licensing option that enables unlimited royalty-free distribution of the report engine for externally facing server and web deployment. http://p.sf.net/sfu/businessobjects _______________________________________________ Monetdb-checkins mailing list Monetdb-checkins@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-checkins

-- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |