March 2010 - developers-list

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/conf monetdb5.conf.in, , 1.86, 1.87
by Martin Kersten 14 Mar '10

14 Mar '10

Fabian Groffen wrote: > On 14-03-2010 08:23:57 +0000, Martin Kersten wrote: >> Update of /cvsroot/monetdb/MonetDB5/conf >> In directory sfp-cvsdas-1.v30.ch3.sourceforge.com:/tmp/cvs-serv1052 >> >> Modified Files: >> monetdb5.conf.in >> Log Message: >> The garbage collector optimizer should be the last in the pipelines. > > Was it really the intention to revert my commit making a separate no > minimal_pipe? If so, then also remove the changelog entry for it, I'd > say. > Also, why is there no garbage collector in the minimal pipe? it should > >> Index: monetdb5.conf.in >> =================================================================== >> RCS file: /cvsroot/monetdb/MonetDB5/conf/monetdb5.conf.in,v >> retrieving revision 1.86 >> retrieving revision 1.87 >> diff -u -d -r1.86 -r1.87 >> --- monetdb5.conf.in 9 Mar 2010 12:02:32 -0000 1.86 >> +++ monetdb5.conf.in 14 Mar 2010 08:23:53 -0000 1.87 >> @@ -255,47 +255,50 @@ >> # optimizers to run, or one or more variables containing the optimizer >> # pipeline to run. The latter is provided for readability purposes >> # only. >> +# >> +# The minimal pipeline necessary by the server to operate correctly: >> +# sql_optimizer=inline,remap,deadcode,multiplex >> >> # Set and define the default pipeline >> sql_optimizer=default_pipe >> >> -# The minimal pipeline necessary by the server to operate correctly >> -minimal_pipe=inline,remap,deadcode,multiplex >> - >> # The default pipe line contains as of Feb2010 mitosis-mergetable-reorder, aimed at large tables and improved access locality >> -default_pipe=inline,remap,evaluate,costModel,coercions,emptySet,aliases,mitosis,mergetable,deadcode,commonTerms,joinPath,reorder,deadcode,reduce,garbageCollector,dataflow,history,multiplex >> +default_pipe=inline,remap,evaluate,costModel,coercions,emptySet,aliases,mitosis,mergetable,deadcode,commonTerms,joinPath,reorder,deadcode,reduce,dataflow,history,multiplex,garbageCollector >> >> # the default pipeline used in the November 2009 release >> -nov2009_pipe=inline,remap,evaluate,costModel,coercions,emptySet,aliases,mergetable,deadcode,constants,commonTerms,joinPath,deadcode,reduce,garbageCollector,dataflow,history,multiplex >> +nov2009_pipe=inline,remap,evaluate,costModel,coercions,emptySet,aliases,mergetable,deadcode,constants,commonTerms,joinPath,deadcode,reduce,dataflow,history,multiplex,garbageCollector >> >> # experimental pipelines stressing various components >> >> # Replication >> -replication_pipe=inline,remap,evaluate,costModel,coercions,emptySet,aliases,mergetable,deadcode,constants,commonTerms,joinPath,deadcode,reduce,garbageCollector,dataflow,history,replication,multiplex >> +replication_pipe=inline,remap,evaluate,costModel,coercions,emptySet,aliases,mergetable,deadcode,constants,commonTerms,joinPath,deadcode,reduce,dataflow,history,replication,multiplex,garbageCollector >> >> # Accumulator >> -accumulator_pipe=inline,remap,evaluate,costModel,coercions,emptySet,aliases,mergetable,deadcode,constants,commonTerms,joinPath,deadcode,reduce,accumulators,garbageCollector,dataflow,history,multiplex >> +accumulator_pipe=inline,remap,evaluate,costModel,coercions,emptySet,aliases,mergetable,deadcode,constants,commonTerms,joinPath,deadcode,reduce,accumulators,dataflow,history,multiplex,garbageCollector >> >> # The Recycle pipeline for Skyserver >> -recycler_pipe=inline,remap,evaluate,costModel,coercions,emptySet,aliases,deadcode,constants,commonTerms,joinPath,deadcode,recycle,reduce,garbageCollector,dataflow,history,multiplex >> +recycler_pipe=inline,remap,evaluate,costModel,coercions,emptySet,aliases,deadcode,constants,commonTerms,joinPath,deadcode,recycle,reduce,dataflow,history,multiplex,garbageCollector >> >> # The Cracking pipeline(s) >> -cracker_pipe=inline,remap,evaluate,costModel,coercions,emptySet,aliases,selcrack,deadcode,constants,commonTerms,joinPath,deadcode,reduce,garbageCollector,dataflow,history,multiplex >> -sidcrack_pipe=inline,remap,evaluate,costModel,coercions,emptySet,aliases,sidcrack,deadcode,constants,commonTerms,joinPath,deadcode,reduce,garbageCollector,dataflow,history,multiplex >> +cracker_pipe=inline,remap,evaluate,costModel,coercions,emptySet,aliases,selcrack,deadcode,constants,commonTerms,joinPath,deadcode,reduce,dataflow,history,multiplex,garbageCollector >> +sidcrack_pipe=inline,remap,evaluate,costModel,coercions,emptySet,aliases,sidcrack,deadcode,constants,commonTerms,joinPath,deadcode,reduce,dataflow,history,multiplex,garbageCollector >> >> # The datacell pipeline >> -datacell_pipe=inline,remap,evaluate,costModel,coercions,emptySet,aliases,deadcode,constants,commonTerms,joinPath,datacell,deadcode,reduce,garbageCollector,dataflow,history,multiplex >> +datacell_pipe=inline,remap,evaluate,costModel,coercions,emptySet,aliases,deadcode,constants,commonTerms,joinPath,datacell,deadcode,reduce,dataflow,history,multiplex,garbageCollector >> >> # The Octopus pipeline for distributed processing (Merovingian enabled >> # platforms only) >> -octopus_pipe=inline,remap,evaluate,costModel,coercions,emptySet,aliases,mitosis,mergetable,deadcode,constants,commonTerms,joinPath,octopus,deadcode,reduce,garbageCollector,dataflow,history,multiplex >> +octopus_pipe=inline,remap,evaluate,costModel,coercions,emptySet,aliases,mitosis,mergetable,deadcode,constants,commonTerms,joinPath,octopus,deadcode,reduce,dataflow,history,multiplex,garbageCollector >> >> # >> #The default + datacyclotron >> -datacyclotron_pipe=inline,remap,evaluate,costModel,coercions,emptySet,aliases,datacyclotron,mergetable,deadcode,constants,commonTerms,joinPath,reorder,deadcode,reduce,garbageCollector,dataflow,history,replication,multiplex >> +datacyclotron_pipe=inline,remap,evaluate,costModel,coercions,emptySet,aliases,datacyclotron,mergetable,deadcode,constants,commonTerms,joinPath,reorder,deadcode,reduce,dataflow,history,replication,multiplex,garbageCollector >> # >> # the default + derivePath >> -derive_pipe=inline,remap,evaluate,costModel,coercions,emptySet,aliases,mergetable,deadcode,constants,commonTerms,derivePath,joinPath,deadcode,reduce,garbageCollector,dataflow,history,multiplex >> +derive_pipe=inline,remap,evaluate,costModel,coercions,emptySet,aliases,mergetable,deadcode,constants,commonTerms,derivePath,joinPath,deadcode,reduce,dataflow,history,multiplex,garbageCollector >> # >> #the default + dictionary >> -dictionary_pipe=inline,remap,dictionary,evaluate,costModel,coercions,emptySet,aliases,mergetable,deadcode,constants,commonTerms,joinPath,deadcode,reduce,garbageCollector,dataflow,history,multiplex >> +dictionary_pipe=inline,remap,dictionary,evaluate,costModel,coercions,emptySet,aliases,mergetable,deadcode,constants,commonTerms,joinPath,deadcode,reduce,dataflow,history,multiplex,garbageCollector >> +# >> +#the default + compression >> +compression_pipe=inline,remap,evaluate,costModel,coercions,emptySet,aliases,mergetable,deadcode,constants,commonTerms,joinPath,deadcode,reduce,dataflow,compression,dataflow,history,multiplex,garbageCollector >> >> >> ------------------------------------------------------------------------------ >> Download Intel® Parallel Studio Eval >> Try the new software tools for yourself. Speed compiling, find bugs >> proactively, and fine-tune applications for parallel performance. >> See why Intel Parallel Studio got high marks during beta. >> http://p.sf.net/sfu/intel-sw-dev >> _______________________________________________ >> Monetdb-checkins mailing list >> Monetdb-checkins(a)lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/monetdb-checkins > > ------------------------------------------------------------------------------ > Download Intel® Parallel Studio Eval > Try the new software tools for yourself. Speed compiling, find bugs > proactively, and fine-tune applications for parallel performance. > See why Intel Parallel Studio got high marks during beta. > http://p.sf.net/sfu/intel-sw-dev > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins

2 2

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB/src/gdk gdk.mx, , 1.310, 1.311 gdk_atoms.mx, , 1.176, 1.177 gdk_bbp.mx, , 1.275, 1.276
by Niels Nes 13 Mar '10

13 Mar '10

On Sat, Mar 13, 2010 at 11:05:37PM +0100, Sjoerd Mullender wrote: > How backward and forward compatible is this? You should probably also > change the GDKLIBRARY value and take care of reading older versions. should be backward compatible, ie if its not there, we have the old situation (at least that was the intention) > > On 2010-03-13 22:48, Niels Nes wrote: > > Update of /cvsroot/monetdb/MonetDB/src/gdk > > In directory sfp-cvsdas-1.v30.ch3.sourceforge.com:/tmp/cvs-serv19213/src/gdk > > > > Modified Files: > > gdk.mx gdk_atoms.mx gdk_bbp.mx > > Log Message: > > LLONG_MAX should be defined before compiling BUNlast (needs this because of > > BUN_NONE) > > > > (keep extra string with options in the BBP.dir, for dynamic storage options) > > > > > > Index: gdk_bbp.mx > > =================================================================== > > RCS file: /cvsroot/monetdb/MonetDB/src/gdk/gdk_bbp.mx,v > > retrieving revision 1.275 > > retrieving revision 1.276 > > diff -u -d -r1.275 -r1.276 > > --- gdk_bbp.mx 7 Mar 2010 19:00:40 -0000 1.275 > > +++ gdk_bbp.mx 13 Mar 2010 21:48:43 -0000 1.276 > > @@ -701,7 +701,7 @@ > > int lastused; > > int nread; > > BATstore *bs; > > - char *s; > > + char *s, *options = NULL; > > char logical[1024]; > > lng inserted, deleted, first, count, capacity; > > unsigned short map_head, map_tail, map_hheap, map_theap; > > @@ -848,8 +848,10 @@ > > @:vheapinit(H)@ > > @:vheapinit(T)@ > > > > - if (buf[nread] != '\n') > > + if (buf[nread] != '\n' && buf[nread] != ' ') > > GDKfatal("BBPinit: invalid format for BBP.dir\n%s", buf); > > + if (buf[nread] == ' ') > > + options = buf+nread+1; > > > > BBP[bid].cache = bs; > > BBP[bid].status = BBPEXISTING; /* do we need other status bits? */ > > @@ -867,6 +869,9 @@ > > else > > BBP[bid].nme[1] = GDKstrdup(BBPtmpname(tailname, sizeof(tailname), -bid)); > > BBP[bid].path = GDKstrdup(filename); > > + BBP[bid].options = NULL; > > + if (options) > > + BBP[bid].options = GDKstrdup(options); > > BBP[bid].lastused = lastused; > > if (lastused > max_stamp) > > max_stamp = lastused; > > @@ -1052,6 +1057,9 @@ > > @:vheapprint(H)@ > > @:vheapprint(T)@ > > > > + if (BBP[i].options) > > + stream_printf(s, " %s", BBP[i].options); > > + > > return stream_printf(s, "\n"); > > } > > > > > > Index: gdk.mx > > =================================================================== > > RCS file: /cvsroot/monetdb/MonetDB/src/gdk/gdk.mx,v > > retrieving revision 1.310 > > retrieving revision 1.311 > > diff -u -d -r1.310 -r1.311 > > --- gdk.mx 10 Mar 2010 22:17:11 -0000 1.310 > > +++ gdk.mx 13 Mar 2010 21:48:38 -0000 1.311 > > @@ -389,6 +389,17 @@ > > # define WIFEXITED(stat_val) (((stat_val) & 255) == 0) > > #endif > > > > +#include <limits.h> /* for *_MIN and *_MAX */ > > +#include <float.h> /* for FLT_MAX and DBL_MAX */ > > +#ifndef LLONG_MAX > > +#ifdef LONGLONG_MAX > > +#define LLONG_MAX LONGLONG_MAX > > +#define LLONG_MIN LONGLONG_MIN > > +#else > > +#define LLONG_MAX LL_CONSTANT(9223372036854775807) > > +#define LLONG_MIN (-LL_CONSTANT(9223372036854775807) - LL_CONSTANT(1)) > > +#endif > > +#endif > > > > #ifdef __cplusplus > > extern "C" { > > @@ -2037,6 +2048,7 @@ > > bat next[2]; /* next BBP slot in link list */ > > BATstore *cache; /* cached header info */ > > str path; /* dir + basename for storage */ > > + str options; /* A string list of options */ > > int refs; /* in-memory references on which the loaded status of a BAT relies */ > > int lrefs; /* logical references on which the existence of a BAT relies */ > > int lastused; /* BBP LRU stamp */ > > @@ -2054,6 +2066,7 @@ > > #define BBP_logical(i) BBP[ABS(i)].nme[(i)<0] > > #define BBP_next(i) BBP[ABS(i)].next[(i)<0] > > #define BBP_physical(i) BBP[ABS(i)].path > > +#define BBP_options(i) BBP[ABS(i)].options > > #define BBP_desc(i) BBP[ABS(i)].cache > > #define BBP_refs(i) BBP[ABS(i)].refs > > #define BBP_lrefs(i) BBP[ABS(i)].lrefs > > > > Index: gdk_atoms.mx > > =================================================================== > > RCS file: /cvsroot/monetdb/MonetDB/src/gdk/gdk_atoms.mx,v > > retrieving revision 1.176 > > retrieving revision 1.177 > > diff -u -d -r1.176 -r1.177 > > --- gdk_atoms.mx 12 Feb 2010 08:58:35 -0000 1.176 > > +++ gdk_atoms.mx 13 Mar 2010 21:48:42 -0000 1.177 > > @@ -718,17 +718,6 @@ > > The routine atomnil returns a pointer to the nil value representation. > > @{ > > @h > > -#include <limits.h> /* for *_MIN and *_MAX */ > > -#include <float.h> /* for FLT_MAX and DBL_MAX */ > > -#ifndef LLONG_MAX > > -#ifdef LONGLONG_MAX > > -#define LLONG_MAX LONGLONG_MAX > > -#define LLONG_MIN LONGLONG_MIN > > -#else > > -#define LLONG_MAX LL_CONSTANT(9223372036854775807) > > -#define LLONG_MIN (-LL_CONSTANT(9223372036854775807) - LL_CONSTANT(1)) > > -#endif > > -#endif > > #define GDK_bit_max ((bit) 1) > > #define GDK_bit_min ((bit) 0) > > #define GDK_chr_max ((chr) SCHAR_MAX) > > > > > > ------------------------------------------------------------------------------ > > Download Intel® Parallel Studio Eval > > Try the new software tools for yourself. Speed compiling, find bugs > > proactively, and fine-tune applications for parallel performance. > > See why Intel Parallel Studio got high marks during beta. > > http://p.sf.net/sfu/intel-sw-dev > > _______________________________________________ > > Monetdb-checkins mailing list > > Monetdb-checkins(a)lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins > > > -- > Sjoerd Mullender > > ------------------------------------------------------------------------------ > Download Intel® Parallel Studio Eval > Try the new software tools for yourself. Speed compiling, find bugs > proactively, and fine-tune applications for parallel performance. > See why Intel Parallel Studio got high marks during beta. > http://p.sf.net/sfu/intel-sw-dev > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- Niels Nes, Centrum Wiskunde & Informatica (CWI) Science Park 123, 1098 XG Amsterdam, The Netherlands room L3.14, phone ++31 20 592-4098 sip:4098@sip.cwi.nl url: http://www.cwi.nl/~niels e-mail: Niels.Nes(a)cwi.nl

1 0

[Monetdb-developers] how to find cost
by Puneet Birwa 11 Mar '10

11 Mar '10

hello, thanks for help, but also can u tell me any way to find the cost of final execution plan, m unable to find any option to see what is the actual cost of final execution plan. The INTERNET now has a personality. YOURS! See your Yahoo! Homepage. http://in.yahoo.com/

2 1

[Monetdb-developers] final execution plan
by Puneet Birwa 11 Mar '10

11 Mar '10

hello, can anyone tell me how in monetdb/xquery we can see what is the final plan with the help of which query is executed i.e how can i find the plan generated at run time for query execution (which tells that which join operation is performed at particular node). The INTERNET now has a personality. YOURS! See your Yahoo! Homepage. http://in.yahoo.com/

2 1

[Monetdb-developers] A SERIOUS CONCURRENCY CONTROL BUG
by oursql 09 Mar '10

09 Mar '10

There is a bug in monetdb, you can run the following script in two clients(mclient.exe), and before test, please disable autocommit option. user A: USER B: CREATE TABLE TEST(C1 INT PRIMARY KEY, C2 INT); INSERT INTO TEST VALUES(1,1); COMMIT; SELECT C2 FROM TEST WHERE C1=1; SELECT C2 FROM TEST WHERE C1=1; UPDATE TEST SET C2=10 WHERE C1=1; UPDATE TEST SET C2=30 WHERE C1=1; COMMIT; COMMIT;--CONFLICT, NEED TO ROLLBACK; ROLLBACK; UPDATE TEST SET C2=30 WHERE C1=1; SELECT C2 FROM TEST WHERE C1=1; --THE RESULT IS 10, NOT 30. TOO BAD! 2010-03-09 oursql

1 0

[Monetdb-developers] monetdb cost model
by anshul gangwar 08 Mar '10

08 Mar '10

hello , Is the cost model used at compile time or at runtime for estimation of cardinalities Your Mail works best with the New Yahoo Optimized IE8. Get it NOW! http://downloads.yahoo.com/in/internetexplorer/

2 1

[Monetdb-developers] Fw: monetdb cost model
by anshul gangwar 08 Mar '10

08 Mar '10

hello , Is the cost model used at compile time or at runtime for estimation of cardinalities please reply as soon as soon as possible Your Mail works best with the New Yahoo Optimized IE8. Get it NOW! http://downloads.yahoo.com/in/internetexplorer/

3 2

[Monetdb-developers] need help in monetdb
by anshul gangwar 08 Mar '10

08 Mar '10

hello how can we get the final executed plan i.e at run time in monetdb. as pysical-algebra-plan only shows the result after compilation time. And at runtime what does it acually does i.e after sampling what does it decide i.e type of join or others thing also. The INTERNET now has a personality. YOURS! See your Yahoo! Homepage. http://in.yahoo.com/

2 1

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/optimizer/Tests ifthencst.stable.out, , 1.3, 1.4
by Stefan Manegold 07 Mar '10

07 Mar '10

FYI: this test has been failing in the Feb2010 branch since Feb 25 2010, i.e., possibly triggered by checkins on Feb 24 2010. http://monetdb.cwi.nl/testing/projects/monetdb/Stable/MonetDB5/.mTests103/G… Note that the output it (now) produces with the Feb2010 branch is different from what been approved here in the CVS head. Stefan On Sun, Mar 07, 2010 at 08:05:00PM +0000, Martin Kersten wrote: > Update of /cvsroot/monetdb/MonetDB5/src/optimizer/Tests > In directory sfp-cvsdas-1.v30.ch3.sourceforge.com:/tmp/cvs-serv3208/Tests > > Modified Files: > ifthencst.stable.out > Log Message: > Approve output > > > Index: ifthencst.stable.out > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/optimizer/Tests/ifthencst.stable.out,v > retrieving revision 1.3 > retrieving revision 1.4 > diff -u -d -r1.3 -r1.4 > --- ifthencst.stable.out 17 Oct 2009 21:22:35 -0000 1.3 > +++ ifthencst.stable.out 7 Mar 2010 20:04:57 -0000 1.4 > @@ -26,8 +26,6 @@ > io.print("never executed"); > exit d; > end tst; > -[ "always executed" ] > -[ "always executed" ] > function user.main():void; > # some frontend generates the following code > mdb.list("user","tst"); > > > ------------------------------------------------------------------------------ > Download Intel® Parallel Studio Eval > Try the new software tools for yourself. Speed compiling, find bugs > proactively, and fine-tune applications for parallel performance. > See why Intel Parallel Studio got high marks during beta. > http://p.sf.net/sfu/intel-sw-dev > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4199 |

1 0

[Monetdb-developers] is there exist any histogram or any other statistic in monetdb/xquery
by Puneet Birwa 07 Mar '10

07 Mar '10

hello, can someone tell me that how monetdb keep track of cardinality of result of query for generating optimal plans or it is completely independent of histogram or anything which keep track of cardinality of result of query. The INTERNET now has a personality. YOURS! See your Yahoo! Homepage. http://in.yahoo.com/

2 1