- developers-list - monetdb.org

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/optimizer opt_derivepath.mx, Feb2009, NONE, 1.1.2.1 Makefile.ag, Feb2009, 1.66, 1.66.2.1 opt_prelude.mx, Feb2009, 1.55.2.3, 1.55.2.4
by Sjoerd Mullender 05 Mar '09

05 Mar '09

How is this not a new feature? Martin Kersten wrote: > Update of /cvsroot/monetdb/MonetDB5/src/optimizer > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv2998/optimizer > > Modified Files: > Tag: Feb2009 > Makefile.ag opt_prelude.mx > Added Files: > Tag: Feb2009 > opt_derivepath.mx > Log Message: > Added the framework for the commissioned derivepath optimizer. > > > U opt_prelude.mx > Index: opt_prelude.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/optimizer/opt_prelude.mx,v > retrieving revision 1.55.2.3 > retrieving revision 1.55.2.4 > diff -u -d -r1.55.2.3 -r1.55.2.4 > --- opt_prelude.mx 1 Mar 2009 19:15:30 -0000 1.55.2.3 > +++ opt_prelude.mx 5 Mar 2009 00:17:12 -0000 1.55.2.4 > @@ -57,6 +57,7 @@ > opt_export str deleteRef; > opt_export str depositRef; > opt_export str deriveRef; > +opt_export str derivePathRef; > opt_export str differenceRef; > opt_export str divRef; > opt_export str disconnectRef; > @@ -216,6 +217,7 @@ > str deleteRef; > str depositRef; > str deriveRef; > +str derivePathRef; > str differenceRef; > str divRef; > str disconnectRef; > @@ -371,6 +373,7 @@ > deleteRef = putName("delete",6); > depositRef = putName("deposit",7); > deriveRef = putName("derive",6); > + derivePathRef = putName("derivePath",10); > differenceRef= putName("difference",10); > divRef = putName("/",1); > disconnectRef= putName("disconnect",10); > > U Makefile.ag > Index: Makefile.ag > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/optimizer/Makefile.ag,v > retrieving revision 1.66 > retrieving revision 1.66.2.1 > diff -u -d -r1.66 -r1.66.2.1 > --- Makefile.ag 18 Jan 2009 11:44:14 -0000 1.66 > +++ Makefile.ag 5 Mar 2009 00:17:12 -0000 1.66.2.1 > @@ -32,7 +32,7 @@ > opt_singleton.mx opt_costModel.mx opt_reduce.mx opt_macro.mx \ > opt_accumulators.mx opt_qep.mx opt_mergetable.mx \ > opt_remoteQueries.mx opt_joinselect.mx opt_partitions.mx \ > - opt_evaluate.mx opt_inline.mx opt_pushranges.mx \ > + opt_evaluate.mx opt_inline.mx opt_pushranges.mx opt_derivepath.mx \ > opt_accessmode.mx opt_joinpath.mx opt_heuristics.mx opt_remap.mx \ > opt_statistics.mx opt_trace.mx opt_recycler.mx opt_dataflow.mx \ > opt_mitosis.mx opt_octopus.mx opt_replicator.mx opt_history.mx > @@ -53,7 +53,7 @@ > opt_singleton.mx opt_costModel.mx opt_reduce.mx opt_macro.mx \ > opt_accumulators.mx opt_qep.mx opt_mergetable.mx \ > opt_remoteQueries.mx opt_joinselect.mx opt_partitions.mx \ > - opt_evaluate.mx opt_inline.mx opt_pushranges.mx \ > + opt_evaluate.mx opt_inline.mx opt_pushranges.mx opt_derivepath.mx \ > opt_accessmode.mx opt_joinpath.mx opt_heuristics.mx opt_remap.mx \ > opt_statistics.mx opt_trace.mx opt_recycler.mx opt_dataflow.mx \ > opt_mitosis.mx opt_octopus.mx opt_replicator.mx opt_history.mx > @@ -72,7 +72,7 @@ > opt_singleton.mx opt_costModel.mx opt_reduce.mx opt_macro.mx \ > opt_accumulators.mx opt_qep.mx opt_mergetable.mx \ > opt_remoteQueries.mx opt_joinselect.mx opt_partitions.mx \ > - opt_evaluate.mx opt_inline.mx opt_pushranges.mx \ > + opt_evaluate.mx opt_inline.mx opt_pushranges.mx opt_derivepath.mx \ > opt_accessmode.mx opt_joinpath.mx opt_heuristics.mx opt_remap.mx \ > opt_statistics.mx opt_trace.mx opt_recycler.mx opt_dataflow.mx \ > opt_mitosis.mx opt_octopus.mx opt_replicator.mx opt_history.mx > > --- NEW FILE: opt_derivepath.mx --- > @' The contents of this file are subject to the MonetDB Public License > @' Version 1.1 (the "License"); you may not use this file except in > @' compliance with the License. You may obtain a copy of the License at > @' http://monetdb.cwi.nl/Legal/MonetDBLicense-1.1.html > @' > @' Software distributed under the License is distributed on an "AS IS" > @' basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the > @' License for the specific language governing rights and limitations > @' under the License. > @' > @' The Original Code is the MonetDB Database System. > @' > @' The Initial Developer of the Original Code is CWI. > @' Portions created by CWI are Copyright (C) 1997-July 2008 CWI. > @' Copyright August 2008-2009 MonetDB B.V. > @' All Rights Reserved. > > @f opt_derivepath > @- Group derive paths > The routine @sc{optimizer.derivepaths()} > walks through the program looking for grouping operations > and cascades them into a single multiple group paths. > To illustrate, consider > @verbatim > (ext1,grp1) := group.new(b); > (ext2,grp2) := group.derive(ext1,grp1, c); > (ext3,grp3) := group.derive(ext2,grp2, d); > @end verbatim > The result becomes. > @verbatim > (ext3,grp3) := group.derivepath(b,c,d); > @end verbatim > @{ > @mal > pattern optimizer.derivePath():str > address OPTderivePath; > pattern optimizer.derivePath(mod:str, fcn:str):str > address OPTderivePath > comment "Join path constructor"; > pattern group.derivePath(l:bat[:any,:any]...):bat[:any,:any] > address ALGderivePath > comment "internal routine to handle join paths. > The type analysis is rather tricky."; > @h > #ifndef _OPT_DERIVEPATH_ > #define _OPT_DERIVEPATH_ > #include "opt_prelude.h" > #include "opt_support.h" > #include "mal_interpreter.h" > > /* #define DEBUG_OPT_DERIVEPATH */ > @c > #include "mal_config.h" > #include "opt_derivepath.h" > > static int > OPTderivePathImplementation(Client cntxt, MalBlkPtr mb, MalStkPtr stk, InstrPtr p) > { > int i, actions=0; > int *pc; > str derivePathRef = putName("derivePath",10); > InstrPtr q; > InstrPtr *old; > int limit,slimit; > > (void) cntxt; > (void) stk; > if (varGetProp(mb, getArg(mb->stmt[0], 0), inlineProp) != NULL) > return 0; > > old= mb->stmt; > limit= mb->stop; > slimit= mb->ssize; > newMalBlkStmt(mb,mb->ssize); > > /* beware, new variables and instructions are introduced */ > pc= (int*) GDKmalloc(sizeof(int)* mb->vtop * 2); /* to find last assignment */ > memset((char*) pc, 0, sizeof(int)* mb->vtop *2); > > for (i = 0; i<limit; i++){ > p= old[i]; > if( getModuleId(p)== groupRef && getFunctionId(p)== newRef ){ > pc[getArg(p,0)] = i; > pc[getArg(p,1)] = i; > } > if( getModuleId(p)== groupRef && getFunctionId(p)== deriveRef ){ > @- > Try to expand its argument list with what we have found so far. > This creates a series of join paths, many of which will be removed during deadcode elimination. > @c > if (pc[getArg(p,2)]== pc[getArg(p,3)]){ > q= copyInstruction(getInstrPtr(mb,pc[getArg(p,2)])); > q= pushArgument(mb,q, getArg(p,5)); > pc[getArg(p,0)] = i; > pc[getArg(p,1)] = i; > setFunctionId(q,derivePathRef); > p= q; > actions++; > #ifdef DEBUG_OPT_DERIVEPATH > stream_printf(cntxt->fdout,"new derivePath instruction\n"); > printInstruction(cntxt->fdout,mb, 0, q, LIST_MAL_ALL); > #endif > } > } > pushInstruction(mb,p); > } > for(; i<slimit; i++) > if(old[i]) > freeInstruction(old[i]); > GDKfree(old); > GDKfree(pc); > OPTDEBUG > if ( actions) > stream_printf(cntxt->fdout,"#opt_derivepath: %d statements glued\n",actions); > return actions; > } > @include optimizerWrapper.mx > @h > @:exportOptimizer(derivePath)@ > #endif > opt_export str ALGderivePath(Client cntxt, MalBlkPtr mb, MalStkPtr stk, InstrPtr pci); > @c > #include "opt_statistics.h" > @:wrapOptimizer(derivePath,OPT_CHECK_ALL)@ > @- > The derive path optimizer takes a derivation sequence and > attempts to minimize the intermediate result. > The choice depends on a good estimate of intermediate > results using properties. > For the time being, we use a simplistic model, based > on the assumption that most joins are foreign key joins anyway. > @c > str > ALGderivePath(Client cntxt, MalBlkPtr mb, MalStkPtr stk, InstrPtr pci) > { > (void) cntxt; > (void) mb; > (void) stk; > (void) pci; > throw(MAL,"group.derivePath","nyi"); > } > @} > > > ------------------------------------------------------------------------------ > Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA > -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise > -Strategies to boost innovation and cut costs with open source participation > -Receive a $600 discount off the registration fee with the source code: SFAD > http://p.sf.net/sfu/XcvMzF8H > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- Sjoerd Mullender

3 2

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/mal mal_debugger.mx, Feb2009, 1.256.2.2, 1.256.2.3
by Stefan Manegold 04 Mar '09

04 Mar '09

[Pre-nighlty testing compilation check service] Fails to compile with icc & gcc with optimization enabled: .../MonetDB5/src/mal/mal_debugger.mx(1501): error #592: variable "p" is used before its value is set mdbCommand(cntxt, mb, stk, p, pc); ^ Stefam On Wed, Mar 04, 2009 at 03:54:07PM +0000, Martin Kersten wrote: > Update of /cvsroot/monetdb/MonetDB5/src/mal > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv10776 > > Modified Files: > Tag: Feb2009 > mal_debugger.mx > Log Message: > The debugger command 'bbp trace <pattern>' has been added to > keep a watch over the lref status of bats in the bbp. > > > U mal_debugger.mx > Index: mal_debugger.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/mal/mal_debugger.mx,v > retrieving revision 1.256.2.2 > retrieving revision 1.256.2.3 > diff -u -d -r1.256.2.2 -r1.256.2.3 > --- mal_debugger.mx 12 Feb 2009 06:34:16 -0000 1.256.2.2 > +++ mal_debugger.mx 4 Mar 2009 15:54:04 -0000 1.256.2.3 > @@ -489,6 +489,7 @@ > mal_export void printBatInfo(stream *f, VarPtr n, ValPtr v); > mal_export void printBatProperties(stream *f, VarPtr n, ValPtr v, str props); > mal_export void printTraceCall(stream *out, MalBlkPtr mb, MalStkPtr stk, int pc, int flags); > +mal_export char BBPTraceCall(Client cntxt, MalBlkPtr mb, MalStkPtr stk, int pc); > mal_export str call2str(MalBlkPtr mb, MalStkPtr stk, int pc, int flags); > > mal_export str runMALDebugger(Client cntxt, Symbol s); > @@ -742,6 +743,63 @@ > stream_printf(out, "#%s\n", msg); > GDKfree(msg); > } > +static void > +mdbBacktrace(Client cntxt, MalStkPtr stk, int pci){ > + for (; stk != NULL; stk = stk->up) { > + printCall(cntxt, stk->blk, stk, pci); > + if (stk->up) > + pci = stk->up->pcup; > + } > +} > +@- > +Sometimes we may want to trace the changes applied to the > +BAT buffer pool and report them together with the MAL > +instruction where it happened. > +@c > +static int BBPtraceEnabled = 0; > +static str BBPtracePattern = NULL; > +char > +BBPTraceCall(Client cntxt, MalBlkPtr mb, MalStkPtr stk, int pc) > +{ > + static int *BBPmirror = NULL; > + static int bbpsize; > + char lbuf[1024], pbuf[1024]; > + int i, action; > + (void) mb; > + > + if ( BBPtraceEnabled == 0) > + return 0; > + if (BBPmirror == NULL ){ > + bbpsize= BBPsize; > + BBPmirror = (int*) GDKzalloc(sizeof(int) * BBPsize); > + } > + if( BBPsize > bbpsize) { > + int *old = BBPmirror; > + BBPmirror = (int*) GDKzalloc(sizeof(int) * BBPsize); > + memcpy((char*)BBPmirror, (char*)old, sizeof(int) * bbpsize); > + bbpsize= BBPsize; > + GDKfree(old); > + } > + /* no growing BBP yet */ > + action = 0; > + for ( i=0; i<bbpsize; i++){ > + /* what happened to this BAT */ > + if ( BBPmirror[i] != BBP_lrefs(i)){ > + BBPlogical(i,lbuf); > + if ( BBPtracePattern && > + strncmp(lbuf,BBPtracePattern,strlen(BBPtracePattern)) ) > + continue; > + BBPphysical(i,pbuf); > + stream_printf(cntxt->fdout, "#BBP state change of %s %s entry %d refs %d:\n", > + lbuf,pbuf,i, BBP_lrefs(i)); > + action = 1; > + } > + BBPmirror[i] = BBP_lrefs(i); > + } > + if ( action ) > + mdbBacktrace(cntxt, stk, pc); > + return 's'; > +} > > static void > printBATproperties(stream *f, BAT *b){ > @@ -756,7 +814,7 @@ > if (b->T->parentid) > stream_printf(f, "tail view on %s ", BBPname(b->T->parentid)); > } > -@+ MAL parser > +@+ MAL debugger parser > The debugger structure is inherited from GDB. > The routine mdbCommand is called with p=0 after finishing a mal- function call > and before continuing at the next level of invocation. > @@ -802,14 +860,6 @@ > return getMalBlkHistory(mb,-1); > } > > -static void > -mdbBacktrace(Client cntxt, MalStkPtr stk, int pci){ > - for (; stk != NULL; stk = stk->up) { > - printCall(cntxt, stk->blk, stk, pci); > - if (stk->up) > - pci = stk->up->pcup; > - } > -} > > void > mdbCommand(Client cntxt, MalBlkPtr mb, MalStkPtr stk, InstrPtr p, int pc) > @@ -1089,6 +1139,16 @@ > int i,limit; > > skipWord(cntxt, b); > + /* bbp change tracing enabling */ > + if ( strncmp(b,"trace",5) == 0){ > + skipWord(cntxt, b); > + BBPtraceEnabled = !BBPtraceEnabled; > + if ( BBPtraceEnabled && *b ) > + BBPtracePattern = GDKstrdup(b); > + stream_printf(out,"#bbp trace enabled %d\n", BBPtraceEnabled); > + stk->cmd = 'c'; > + break; > + } > i = BBPindex(b); > if( i) > limit=i+1; > @@ -1437,6 +1497,8 @@ > return; > } > > + if ( BBPTraceCall(cntxt,mb,stk,pc)) > + mdbCommand(cntxt, mb, stk, p, pc); > switch (stk->cmd) { > case 'c': > p = getInstrPtr(mb, pc); > > > ------------------------------------------------------------------------------ > Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA > -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise > -Strategies to boost innovation and cut costs with open source participation > -Receive a $600 discount off the registration fee with the source code: SFAD > http://p.sf.net/sfu/XcvMzF8H > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

Re: [Monetdb-developers] [Monetdb-bugs] [ monetdb-Bugs-2654300 ] problem with views reusing aliases ?
by Romulo Goncalves 02 Mar '09

02 Mar '09

SourceForge.net wrote: > Bugs item #2654300, was opened at 2009-03-02 12:41 > Message generated for change (Tracker Item Submitted) made by Item Submitter > You can respond by visiting: > https://sourceforge.net/tracker/?func=detail&atid=482468&aid=2654300&group_… > > Please note that this message will contain a full copy of the comment thread, > including the initial issue submission, for this request, > not just the latest update. > Category: SQL/Core > Group: SQL "stable" > Status: Open > Resolution: None > Priority: 5 > Private: No > Submitted By: Roberto Cornacchia (cornuz) > Assigned to: Niels Nes (nielsnes) > Summary: problem with views reusing aliases ? > > Initial Comment: > [ on Feb2009 branch ] > > create table x (n int, s varchar(10)); > insert into x values (1, 'one'); > insert into x values (2, 'two'); > insert into x values (3, 'three'); > > -- swap the two attributes > create view x1 as > select s as a1, n as a2 from x; > > -- swap them again > create view x2 as > select a2 as a1, a1 as a2 from x1; > > -- select on the original 's' attribute fails > select * from x2 where a2='two'; > > This is the output: > > $ mclient -lsql -dtest02 -G < bug2.sql The algebra version is the default one in the current as well as in the stable branch... So how the -G option is still valid? Romulo > [ 1 ] > [ 1 ] > [ 1 ] > MAPI = monetdb@localhost:50000 > QUERY = select * from x2 where a2='two'; > ERROR = !TypeException:user.s0_1[4]:'algebra.uselect' undefined in: _9:any := algebra.uselect(_2:bat[:oid,:str], _8:int) > !TypeException:user.s0_1[6]:'algebra.kdifference' undefined in: _12:any := algebra.kdifference(_9:any, _10:bat[:oid,:str]) > !TypeException:user.s0_1[7]:'algebra.uselect' undefined in: _13:any := algebra.uselect(_10:bat[:oid,:str], _8:int) > !TypeException:user.s0_1[8]:'algebra.kunion' undefined in: _14:any := algebra.kunion(_12:any, _13:any) > !TypeException:user.s0_1[10]:'algebra.uselect' undefined in: _17:any := algebra.uselect(_15:bat[:oid,:str], _8:int) > !TypeException:user.s0_1[11]:'algebra.kunion' undefined in: _18:any := algebra.kunion(_14:any, _17:any) > !TypeException:user.s0_1[14]:'algebra.kdifference' undefined in: _21:any := algebra.kdifference(_18:any, _20:bat[:oid,:oid]) > !TypeException:user.s0_1[16]:'algebra.markT' undefined in: _24:any := algebra.markT(_21:any, _22:oid) > !TypeException:user.s0_1[17]:'bat.reverse' undefined in: _25:any := bat.reverse(_24:any) > !TypeException:user.s0_1[24]:'algebra.join' undefined in: _33:any := algebra.join(_25:any, _32:bat[:oid,:int]) > !TypeException:user.s0_1[28]:'algebra.join' undefined in: _37:any := algebra.join(_25:any, _36:bat[:oid,:str]) > !SQLException:int:conversion of string 'two' failed > > > ---------------------------------------------------------------------- > > You can respond by visiting: > https://sourceforge.net/tracker/?func=detail&atid=482468&aid=2654300&group_… > > ------------------------------------------------------------------------------ > Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA > -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise > -Strategies to boost innovation and cut costs with open source participation > -Receive a $600 discount off the registration fee with the source code: SFAD > http://p.sf.net/sfu/XcvMzF8H > _______________________________________________ > Monetdb-bugs mailing list > Monetdb-bugs(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-bugs

1 0

Re: [Monetdb-developers] [Monetdb-sql-checkins] sql/src/common sql_types.mx, Feb2009, 1.121.2.1, 1.121.2.2
by Stefan Manegold 27 Feb '09

27 Feb '09

FYI: This checkin seems to change the DB schema, making existing databases incompatible with the code after this change: !FATAL: SQL schema missing or incompatible, rebuild from archive Stefan On Thu, Feb 26, 2009 at 04:56:32PM +0000, Roberto Cornacchia wrote: > Update of /cvsroot/monetdb/sql/src/common > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv29270/common > > Modified Files: > Tag: Feb2009 > sql_types.mx > Log Message: > added prod() aggregation function in SQL > > Not part of standard SQL, but useful and available in most SQL DBs > > > > U sql_types.mx > Index: sql_types.mx > =================================================================== > RCS file: /cvsroot/monetdb/sql/src/common/sql_types.mx,v > retrieving revision 1.121.2.1 > retrieving revision 1.121.2.2 > diff -u -d -r1.121.2.1 -r1.121.2.2 > --- sql_types.mx 11 Feb 2009 19:35:44 -0000 1.121.2.1 > +++ sql_types.mx 26 Feb 2009 16:56:29 -0000 1.121.2.2 > @@ -1361,6 +1361,35 @@ > t++; /* LNG */ > sql_create_aggr("sum", "aggr", "sum", *(t), *(t)); > > + /* prod for numerical and decimals */ > + sql_create_aggr("prod", "aggr", "product", BTE, SHT); > + sql_create_aggr("prod", "aggr", "product", BTE, INT); > + sql_create_aggr("prod", "aggr", "product", BTE, WRD); > + sql_create_aggr("prod", "aggr", "product", BTE, LNG); > + > + sql_create_aggr("prod", "aggr", "product", SHT, INT); > + sql_create_aggr("prod", "aggr", "product", SHT, WRD); > + sql_create_aggr("prod", "aggr", "product", SHT, LNG); > + > + sql_create_aggr("prod", "aggr", "product", INT, WRD); > + sql_create_aggr("prod", "aggr", "product", INT, LNG); > + > + sql_create_aggr("prod", "aggr", "product", WRD, WRD); > + sql_create_aggr("prod", "aggr", "product", WRD, LNG); > + sql_create_aggr("prod", "aggr", "product", LNG, LNG); > + > + t = decimals; /* BTE */ > + sql_create_aggr("prod", "aggr", "product", *(t), *(t+1)); > + sql_create_aggr("prod", "aggr", "product", *(t), *(t+2)); > + sql_create_aggr("prod", "aggr", "product", *(t), *(t+3)); > + t++; /* SHT */ > + sql_create_aggr("prod", "aggr", "product", *(t), *(t+1)); > + sql_create_aggr("prod", "aggr", "product", *(t), *(t+2)); > + t++; /* INT */ > + sql_create_aggr("prod", "aggr", "product", *(t), *(t+1)); > + t++; /* LNG */ > + sql_create_aggr("prod", "aggr", "product", *(t), *(t)); > + > for (t = numerical; t < floats; t++) { > sql_create_aggr("avg", "aggr", "avg", *(t), DBL); > sql_create_func("mod", "calc", "%", *t, *t, *t, SCALE_FIX); > @@ -1368,6 +1397,7 @@ > > for (t = floats; t < dates; t++) { > sql_create_aggr("sum", "aggr", "sum", *t, *t); > + sql_create_aggr("prod", "aggr", "product", *t, *t); > sql_create_aggr("avg", "aggr", "avg", *t, DBL); > > sql_create_func("mod", "calc", "fmod", *t, *t, *t, SCALE_FIX); > > > ------------------------------------------------------------------------------ > Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA > -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise > -Strategies to boost innovation and cut costs with open source participation > -Receive a $600 discount off the registration fee with the source code: SFAD > http://p.sf.net/sfu/XcvMzF8H > _______________________________________________ > Monetdb-sql-checkins mailing list > Monetdb-sql-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-sql-checkins -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 1

Re: [Monetdb-developers] [Monetdb-sql-checkins] sql/src/common sql_types.mx, Feb2009, 1.121.2.2, 1.121.2.3
by Stefan Manegold 27 Feb '09

27 Feb '09

On Thu, Feb 26, 2009 at 11:47:57PM +0000, Roberto Cornacchia wrote: > Update of /cvsroot/monetdb/sql/src/common > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv3519/common > > Modified Files: > Tag: Feb2009 > sql_types.mx > Log Message: > reverting addition of prob() aggregation, as it would break db compatibility Thanks! (In particular for properly doing it using CVS commands, only!) I prevented propagation of this "undo" to the developlment trunk; the "do" had already been propagated yesterday; hence, the prod() aggregation is now avilable in the development trunk. Stefan > > > U sql_types.mx > Index: sql_types.mx > =================================================================== > RCS file: /cvsroot/monetdb/sql/src/common/sql_types.mx,v > retrieving revision 1.121.2.2 > retrieving revision 1.121.2.3 > diff -u -d -r1.121.2.2 -r1.121.2.3 > --- sql_types.mx 26 Feb 2009 16:56:29 -0000 1.121.2.2 > +++ sql_types.mx 26 Feb 2009 23:47:55 -0000 1.121.2.3 > @@ -1361,35 +1361,6 @@ > t++; /* LNG */ > sql_create_aggr("sum", "aggr", "sum", *(t), *(t)); > > - /* prod for numerical and decimals */ > - sql_create_aggr("prod", "aggr", "product", BTE, SHT); > - sql_create_aggr("prod", "aggr", "product", BTE, INT); > - sql_create_aggr("prod", "aggr", "product", BTE, WRD); > - sql_create_aggr("prod", "aggr", "product", BTE, LNG); > - > - sql_create_aggr("prod", "aggr", "product", SHT, INT); > - sql_create_aggr("prod", "aggr", "product", SHT, WRD); > - sql_create_aggr("prod", "aggr", "product", SHT, LNG); > - > - sql_create_aggr("prod", "aggr", "product", INT, WRD); > - sql_create_aggr("prod", "aggr", "product", INT, LNG); > - > - sql_create_aggr("prod", "aggr", "product", WRD, WRD); > - sql_create_aggr("prod", "aggr", "product", WRD, LNG); > - sql_create_aggr("prod", "aggr", "product", LNG, LNG); > - > - t = decimals; /* BTE */ > - sql_create_aggr("prod", "aggr", "product", *(t), *(t+1)); > - sql_create_aggr("prod", "aggr", "product", *(t), *(t+2)); > - sql_create_aggr("prod", "aggr", "product", *(t), *(t+3)); > - t++; /* SHT */ > - sql_create_aggr("prod", "aggr", "product", *(t), *(t+1)); > - sql_create_aggr("prod", "aggr", "product", *(t), *(t+2)); > - t++; /* INT */ > - sql_create_aggr("prod", "aggr", "product", *(t), *(t+1)); > - t++; /* LNG */ > - sql_create_aggr("prod", "aggr", "product", *(t), *(t)); > - > for (t = numerical; t < floats; t++) { > sql_create_aggr("avg", "aggr", "avg", *(t), DBL); > sql_create_func("mod", "calc", "%", *t, *t, *t, SCALE_FIX); > @@ -1397,7 +1368,6 @@ > > for (t = floats; t < dates; t++) { > sql_create_aggr("sum", "aggr", "sum", *t, *t); > - sql_create_aggr("prod", "aggr", "product", *t, *t); > sql_create_aggr("avg", "aggr", "avg", *t, DBL); > > sql_create_func("mod", "calc", "fmod", *t, *t, *t, SCALE_FIX); > > > ------------------------------------------------------------------------------ > Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA > -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise > -Strategies to boost innovation and cut costs with open source participation > -Receive a $600 discount off the registration fee with the source code: SFAD > http://p.sf.net/sfu/XcvMzF8H > _______________________________________________ > Monetdb-sql-checkins mailing list > Monetdb-sql-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-sql-checkins -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 2

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/optimizer opt_accumulators.mx, Feb2009, 1.32, 1.32.2.1 opt_aliases.mx, Feb2009, 1.24, 1.24.2.1 opt_coercion.mx, Feb2009, 1.14, 1.14.2.1 opt_commonTerms.mx, Feb2009, 1.49, 1.49.2.1 opt_costModel.mx, Feb2009, 1.39, 1.39.2.1 opt_dataflow.mx, Feb2009, 1.21, 1.21.2.1 opt_deadcode.mx, Feb2009, 1.40, 1.40.2.1 opt_emptySet.mx, Feb2009, 1.54, 1.54.2.1 opt_evaluate.mx, Feb2009, 1.22, 1.22.2.1 opt_garbageCollector.mx, Feb2009, 1.45.2.2, 1.45.2.3 opt_history.mx,
by Stefan Manegold 25 Feb '09

25 Feb '09

On Wed, Feb 25, 2009 at 02:19:44PM +0000, Martin Kersten wrote: > Update of /cvsroot/monetdb/MonetDB5/src/optimizer > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv25174 > > Modified Files: > Tag: Feb2009 > opt_accumulators.mx opt_aliases.mx opt_coercion.mx > opt_commonTerms.mx opt_costModel.mx opt_dataflow.mx > opt_deadcode.mx opt_emptySet.mx opt_evaluate.mx > opt_garbageCollector.mx opt_history.mx opt_inline.mx > opt_joinpath.mx opt_mergetable.mx opt_mitosis.mx > opt_multiplex.mx opt_prelude.mx opt_reduce.mx opt_remap.mx > optimizer.mx > Log Message: > Give a summary of the optimizer decisions and effects when --optimizers is > used. > [...] > > U opt_commonTerms.mx > Index: opt_commonTerms.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/optimizer/opt_commonTerms.mx,v > retrieving revision 1.49 > retrieving revision 1.49.2.1 > diff -u -d -r1.49 -r1.49.2.1 > --- opt_commonTerms.mx 27 Jan 2009 22:29:48 -0000 1.49 > +++ opt_commonTerms.mx 25 Feb 2009 14:19:41 -0000 1.49.2.1 > @@ -244,6 +244,9 @@ > if( old[i]) > freeInstruction(old[i]); > GDKfree(old); > + PROPDEBUG ^^^^ shouldn't this one read "OPTDEBUG" ? Stefan > + if (actions) > + stream_printf(cntxt->fdout,"#opt_commonTerms: %d statemtents catched\n",actions); > return actions; > } > [...] -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/optimizer opt_garbageCollector.mx, Feb2009, 1.45, 1.45.2.1
by Stefan Manegold 25 Feb '09

25 Feb '09

I could isolate this checkin as being the cuase that made most SQL tests segfault during testin glast night. Stefan On Tue, Feb 24, 2009 at 11:17:19PM +0000, Martin Kersten wrote: > Update of /cvsroot/monetdb/MonetDB5/src/optimizer > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv25714 > > Modified Files: > Tag: Feb2009 > opt_garbageCollector.mx > Log Message: > Keep the optimizer statements. > > > U opt_garbageCollector.mx > Index: opt_garbageCollector.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/optimizer/opt_garbageCollector.mx,v > retrieving revision 1.45 > retrieving revision 1.45.2.1 > diff -u -d -r1.45 -r1.45.2.1 > --- opt_garbageCollector.mx 7 Jan 2009 14:15:13 -0000 1.45 > +++ opt_garbageCollector.mx 24 Feb 2009 23:17:16 -0000 1.45.2.1 > @@ -218,8 +218,11 @@ > } > #endif > for(; i<slimit; i++) > + if( old[i] == pci) > + freeInstruction(pci); > + else > if( old[i]) > - freeInstruction(old[i]); > + pushInstruction(mb,old[i]); > GDKfree(old); > GDKfree(span); > return actions; > > > ------------------------------------------------------------------------------ > Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA > -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise > -Strategies to boost innovation and cut costs with open source participation > -Receive a $600 discount off the registration fee with the source code: SFAD > http://p.sf.net/sfu/XcvMzF8H > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

[Monetdb-developers] monetdb-client for python doesn't seem to work for booleans.
by Vikram Bhandoh 23 Feb '09

23 Feb '09

Hi I've been working on a django backend for monetdb. For people who are interested in it have a look at it http://github.com/vikram/monetdb/tree/master I found a couple of issues with the python monetdb-client which are important for python. It doesn't seem to support boolean fields and doesn't give the lastrowid. I have a patch here for both. I'm sure about the lastrowid. As I haven't tested it in anger, could anyone tell me if this is alright or have I got things completely wrong. Thanks Vikram diff MonetSQLdb/converters.py ./converters.py 120,125d119 < def Boolean2Str(x, d): < if x: < return 'TRUE' < else: < return 'FALSE' < 144d137 < types.BooleanType: Boolean2Str, diff MonetSQLdb/cursors.py ./cursors.py 38d37 < self.lastrowid = 0 82d80 < 132,134d129 < if self._result.lastid()>0: < self.lastrowid = self._result.lastid() <

2 1

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/runtime shredder.mx, Feb2009, 1.153.2.5, 1.153.2.6
by Stefan Manegold 20 Feb '09

20 Feb '09

On Fri, Feb 20, 2009 at 02:18:38PM +0000, Jan Flokstra wrote: > Update of /cvsroot/monetdb/pathfinder/runtime > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv16245 > > Modified Files: > Tag: Feb2009 > shredder.mx > Log Message: > - fixed shred_characters() bug, sorry :-( Testing suggests that this indeed fixes the previously reported errors / failing tests, again. Thanks! We would *VERY MUCH APPRECIATE*, though, if you could run Mtest yourself --- and check the results to ensure that your changes do not break previously working code & tests --- *before* checking in, at least when bugfixing on the stable branch just before a planned release... Running Mtest costs (each of) you just 12 key strokes (`Mtest.py -r[Enter]`) and (even for pathfinder) takes much less time than it takes us to sort out the problems of many checkins by different people, and Sjoerd to re-build the release candidates more often than necessary ... (Once again) thank you very much in advance for you cooperation! Stefan > > > U shredder.mx > Index: shredder.mx > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/runtime/shredder.mx,v > retrieving revision 1.153.2.5 > retrieving revision 1.153.2.6 > diff -u -d -r1.153.2.5 -r1.153.2.6 > --- shredder.mx 20 Feb 2009 13:06:50 -0000 1.153.2.5 > +++ shredder.mx 20 Feb 2009 14:18:35 -0000 1.153.2.6 > @@ -1105,9 +1105,20 @@ > const xmlChar *cs, > int n) > { > +#ifdef OLD_SHRED_CHARACTERS > shredCtxStruct *shredCtx = (shredCtxStruct*) xmlCtx; > + int l = MIN(PFSHRED_STRLEN_MAX -(int) shredCtx->content, n); > > - if ( (shredCtx->content + n + 1) < shredCtx->content_max) { > + memcpy(&(shredCtx->content_buf[shredCtx->content]), cs, l); > + shredCtx->content += l; > + > + if (l < n) > + stream_printf(GDKout, "!WARNING: shred_characters truncated text node > %d characters(starts with `%.16s...')\n", > + (int) PFSHRED_STRLEN_MAX, cs); > +#else > + shredCtxStruct *shredCtx = (shredCtxStruct*) xmlCtx; > + > + if ( (shredCtx->content + n + 1) > shredCtx->content_max) { > shredCtx->content_max = shredCtx->content + n + 1; > shredCtx->content_buf = GDKrealloc(shredCtx->content_buf, shredCtx->content_max); > if (shredCtx->content_buf == NULL) { > @@ -1117,6 +1128,7 @@ > } > memcpy(&(shredCtx->content_buf[shredCtx->content]), cs, n); > shredCtx->content += n; > +#endif > } > > /** > > > ------------------------------------------------------------------------------ > Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA > -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise > -Strategies to boost innovation and cut costs with open source participation > -Receive a $600 discount off the registration fee with the source code: SFAD > http://p.sf.net/sfu/XcvMzF8H > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins > > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/runtime shredder.mx, Feb2009, 1.153.2.3, 1.153.2.4
by Stefan Manegold 20 Feb '09

20 Feb '09

Jan, unfortunately, this checkin breaks more than 30 pathfinder tests: benchmarks/XBench/TC/MD/shred_docs.milS benchmarks/XBench/TC/MD/load_docs.xq benchmarks/XBench/TC/MD/count_nodes.xq benchmarks/XBench/TC/MD/q01.xq ... benchmarks/XBench/TC/MD/q19.xq benchmarks/XMach-1/load_docs.xq benchmarks/XMach-1/count_nodes.xq benchmarks/XMach-1/q1.xq ... benchmarks/XMach-1/q8.xq tests/BugTracker/immune_for_updates.SF-1766259.xq tests/BugTracker/immune_for_updates.SF-1981852.xq Stefan On Fri, Feb 20, 2009 at 09:06:47AM +0000, Jan Flokstra wrote: > Update of /cvsroot/monetdb/pathfinder/runtime > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv26141 > > Modified Files: > Tag: Feb2009 > shredder.mx > Log Message: > - fix [ monetdb-Bugs-2586088 ] XQ: large text nodes > > > > U shredder.mx > Index: shredder.mx > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/runtime/shredder.mx,v > retrieving revision 1.153.2.3 > retrieving revision 1.153.2.4 > diff -u -d -r1.153.2.3 -r1.153.2.4 > --- shredder.mx 6 Feb 2009 22:52:30 -0000 1.153.2.3 > +++ shredder.mx 20 Feb 2009 09:06:45 -0000 1.153.2.4 > @@ -362,6 +362,7 @@ > int sp; /* the stackpointer */ > /* buffer administration */ > int content; /* content ptr in char buff */ > + int content_max; /* max size of content buffer */ > char *content_buf; /* character buffer */ > /* */ > size_t fileSize; /* size of the input file */ > @@ -1105,14 +1106,13 @@ > int n) > { > shredCtxStruct *shredCtx = (shredCtxStruct*) xmlCtx; > - int l = MIN(PFSHRED_STRLEN_MAX -(int) shredCtx->content, n); > - > - memcpy(&(shredCtx->content_buf[shredCtx->content]), cs, l); > - shredCtx->content += l; > > - if (l < n) > - stream_printf(GDKout, "!WARNING: shred_characters truncated text node > %d characters(starts with `%.16s...')\n", > - (int) PFSHRED_STRLEN_MAX, cs); > + if ( (shredCtx->content + n + 1) < shredCtx->content_max) { > + shredCtx->content_max = shredCtx->content + n + 1; > + shredCtx->content_buf = GDKrealloc(shredCtx->content_buf, shredCtx->content_max); > + } > + memcpy(&(shredCtx->content_buf[shredCtx->content]), cs, n); > + shredCtx->content += n; > } > > /** > @@ -1831,7 +1831,8 @@ > /* allocate some buffers used during parse */ > shredCtx->val = (char*) GDKmalloc(PFSHRED_BUFLEN+1); > shredCtx->lifo = (node_t *) GDKmalloc((XML_DEPTH_MAX+1) * sizeof(node_t)); > - shredCtx->content_buf = (char*) GDKmalloc(PFSHRED_STRLEN_MAX); > + shredCtx->content_max = PFSHRED_STRLEN_MAX; > + shredCtx->content_buf = (char*) GDKmalloc(shredCtx->content_max); > if (shredCtx->val == NULL || shredCtx->content_buf == NULL || shredCtx->lifo == NULL) { > return shredder_free(shredCtx,-shredCtx->incremental); > } > > > ------------------------------------------------------------------------------ > Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA > -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise > -Strategies to boost innovation and cut costs with open source participation > -Receive a $600 discount off the registration fee with the source code: SFAD > http://p.sf.net/sfu/XcvMzF8H > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 1