- developers-list - monetdb.org

[Monetdb-developers] HOST_NAME_MAX op mac os 10.5
by Ying Zhang 29 May '08

29 May '08

Dear all, I use HOST_NAME_MAX in my code. Compiling the code on a CWI machine does not have any problem, however, compilation on Mac OS 10.5 complains that HOST_NAME_MAX is not defined. fgrep on mac 10.5 says: $fgrep -r HOST_NAME_MAX /usr/include/* /usr/include/limits.h:#define _POSIX_HOST_NAME_MAX 255 /usr/include/unistd.h:#define _SC_HOST_NAME_MAX 72 How (and where) can I add proper check to define HOST_NAME_MAX for darwin 9? Thanks! Jennie

3 3

Re: [Monetdb-developers] Error in compilation of PF-ROX
by Peter Boncz 28 May '08

28 May '08

Hi Stefan, Riham PF_ROX has the new indexing code, that Lefteris has tried to describe in his recent paper. This code was only checked for correct operation from MIL in the code generated by the pf-rox java program of Riham. The correct operation of the new indexing code when activated automatically from mps was not tested yet. It may well be the source of trouble. i will pick this up once I have some time again (..). Peter Dear all, I checked all (modified) files in the PF_ROX branch by hand, and all but runtime/pathfinder.mx & runtime/pf_support.mx (this means in particular the compiler/mil/milprint_summer.c I fixed the other day) are OK, i.e., they contain all (intended) changes applied in the PF_ROX branch as well as all chnages that were made on the development trunk since the creation of the PF_ROX branch (and hence properly propagated). For runtime/pathfinder.mx & runtime/pf_support.mx, I am not yet sure, whether everything is OK. There have been numerous "interleaved" (and partly "redundant" or "conflicting") changes both genuine on the PF_ROX branch and propagated from the development trunk, and the (very tedious) analysis requires more time than I could spent today. More to come "soon", I hope ... Unfortunately, while only 50 (100) tests fail in the new stable branch (and in the development trunk), more than 500 tests seem to fail in the PF_ROX branch --- also this requires "some" more ananlysis ... Stefan

1 0

[Monetdb-developers] Error in compilation of PF-ROX
by Riham Abdel Kader 28 May '08

28 May '08

Hi all, I'm getting the following error while compiling PF-ROX: make[5]: Entering directory `/local/kader/home/workspace_pathfinder/MonetDB_XQuery/pathfinder/compiler/mil' /bin/sh ../../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I. -I../.. -I. -I../include -I./../include -I/local/kader/home/workspace_pathfinder/MonetDB_XQuery/include/MonetDB -I/local/kader/home/workspace_pathfinder/MonetDB_XQuery/include/MonetDB/common -I/local/kader/home/workspace_pathfinder/MonetDB_XQuery/include/MonetDB/gdk -DLIBMIL -g -O2 -Wall -Wextra -std=c99 -Werror-implicit-function-declaration -Werror -Wpointer-arith -Wdeclaration-after-statement -Wundef -Wp,-D_FORTIFY_SOURCE=2 -Wno-declaration-after-statement -c -o libmil_la-milprint_summer.lo `test -f 'milprint_summer.c' || echo './'`milprint_summer.c gcc -DHAVE_CONFIG_H -I. -I. -I../.. -I. -I../include -I./../include -I/local/kader/home/workspace_pathfinder/MonetDB_XQuery/include/MonetDB -I/local/kader/home/workspace_pathfinder/MonetDB_XQuery/include/MonetDB/common -I/local/kader/home/workspace_pathfinder/MonetDB_XQuery/include/MonetDB/gdk -DLIBMIL -g -O2 -Wall -Wextra -std=c99 -Werror-implicit-function-declaration -Werror -Wpointer-arith -Wdeclaration-after-statement -Wundef -Wp,-D_FORTIFY_SOURCE=2 -Wno-declaration-after-statement -c milprint_summer.c -fPIC -DPIC -o .libs/libmil_la-milprint_summer.o milprint_summer.c: In function 'translateFunction': milprint_summer.c:8518: error: expected expression before 'else' make[5]: *** [libmil_la-milprint_summer.lo] Error 1 make[5]: Leaving directory `/local/kader/home/workspace_pathfinder/MonetDB_XQuery/pathfinder/compiler/mil' make[4]: *** [install] Error 2 make[4]: Leaving directory `/local/kader/home/workspace_pathfinder/MonetDB_XQuery/pathfinder/compiler/mil' make[3]: *** [install-recursive] Error 1 make[3]: Leaving directory `/local/kader/home/workspace_pathfinder/MonetDB_XQuery/pathfinder/compiler' make[2]: *** [install] Error 2 make[2]: Leaving directory `/local/kader/home/workspace_pathfinder/MonetDB_XQuery/pathfinder/compiler' make[1]: *** [install-recursive] Error 1 make[1]: Leaving directory `/local/kader/home/workspace_pathfinder/MonetDB_XQuery/pathfinder' make: *** [install] Error 2 I checked out all needed modules yesterday night. Best regards, Riham -- Address: Database Group, Dept of EEMCS, University of Twente, POBox 217, 7500 AE Enschede, The Netherlands Phone: +31 (0)53 489 4628 Fax: +31 (0)53 489 2927 Email: r.abdelkader(a)ewi.utwente.nl

3 4

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/mal/Tests All, MonetDB_5-6, 1.97, 1.97.2.1
by Stefan Manegold 27 May '08

27 May '08

Propagation has been prevented. Stefan On Tue, May 27, 2008 at 04:51:01PM +0000, Romulo Goncalves wrote: > Update of /cvsroot/monetdb/MonetDB5/src/mal/Tests > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv7156 > > Modified Files: > Tag: MonetDB_5-6 > All > Log Message: > If the recycle is still in developing, so the tests should be deactivated on the stable testing... > > Note:Do not progate this change to the current branch > > > U All > Index: All > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/mal/Tests/All,v > retrieving revision 1.97 > retrieving revision 1.97.2.1 > diff -u -d -r1.97 -r1.97.2.1 > --- All 22 Mar 2008 08:55:24 -0000 1.97 > +++ All 27 May 2008 16:50:59 -0000 1.97.2.1 > @@ -222,10 +222,10 @@ > tst901b > # > > -recycle00 > -recycle01 > -recycle02 > -recycle03 > +#recycle00 > +#recycle01 > +#recycle02 > +#recycle03 > venks > > saveload > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins > > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

Re: [Monetdb-developers] [Monetdb-sql-checkins] sql/src/test/Tests All, SQL_2-24, 1.51, 1.51.2.1
by Stefan Manegold 27 May '08

27 May '08

Propagation has been prevented. Stefan On Tue, May 27, 2008 at 04:48:00PM +0000, Romulo Goncalves wrote: > Update of /cvsroot/monetdb/sql/src/test/Tests > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv5851 > > Modified Files: > Tag: SQL_2-24 > All > Log Message: > Deactivated > crashme > 50ways > > These are tests to be tested on the current branch and not in the stable. > In the previous release we had the same procedure... > > Note: Do not propagated this change to the current one... > > > U All > Index: All > =================================================================== > RCS file: /cvsroot/monetdb/sql/src/test/Tests/All,v > retrieving revision 1.51 > retrieving revision 1.51.2.1 > diff -u -d -r1.51 -r1.51.2.1 > --- All 6 Mar 2008 08:33:16 -0000 1.51 > +++ All 27 May 2008 16:47:58 -0000 1.51.2.1 > @@ -88,8 +88,8 @@ > setoptimizer > load_dec_as_int > string > -NOT_WIN32?crashme > -NOT_WIN32?50ways > +#NOT_WIN32?crashme > +#NOT_WIN32?50ways > load_with_offset > copy_into > antiselect > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-sql-checkins mailing list > Monetdb-sql-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-sql-checkins > > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/optimizer opt_recycler.mx, , 1.23, 1.24
by Stefan Manegold 27 May '08

27 May '08

Milena, is this a (bug-?) "fix" that should (have) go(ne) into the upcoming release (and hence release branch)? Stefan On Tue, May 27, 2008 at 09:37:31AM +0000, Milena Ivanova wrote: > Update of /cvsroot/monetdb/MonetDB5/src/optimizer > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv17027/optimizer > > Modified Files: > opt_recycler.mx > Log Message: > Fixed chain of recycled instructions. > > > U opt_recycler.mx > Index: opt_recycler.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/optimizer/opt_recycler.mx,v > retrieving revision 1.23 > retrieving revision 1.24 > diff -u -d -r1.23 -r1.24 > --- opt_recycler.mx 20 May 2008 07:20:19 -0000 1.23 > +++ opt_recycler.mx 27 May 2008 09:37:29 -0000 1.24 > @@ -160,17 +160,18 @@ > if( getFunctionId(p)== selectRef && recycled[getArg(p,1)] ) > p->recycle = -1; > if( getFunctionId(p)== uselectRef && recycled[getArg(p,1)] && > - !ATOMvarsized( getGDKType( getArgType(mb,p,1))) > - ){ > - q= copyInstruction(p); > - getArg(q,0)= newTmpVariable(mb,TYPE_any); > - setFunctionId(q, selectRef); > + !ATOMvarsized( getGDKType( getArgType(mb,p,1)))){ > + q = copyInstruction(p); > + getArg(q,0)= newTmpVariable(mb,TYPE_any); > + setFunctionId(q, selectRef); > q->recycle = -1; > + recycled[getArg(q,0)] = 1; > pushInstruction(mb,q); > - getArg(p,1)= getArg(q,0); > + getArg(p,1) = getArg(q,0); > setFunctionId(p,markTRef); > - p->argc=2; > - p->recycle = -1; > + p->argc = 2; > + p->recycle = -1; > + recycled[getArg(p,0)] = 1; > } > @- > The sql.bind instructions should be handled carefully > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins > > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

3 2

[Monetdb-developers] best practices with connection pool
by Stefan de Konink 27 May '08

27 May '08

Hi, Maybe I should move this to the users part, if so, please inform me. What is currently the best practice of connecting to MonetDB5/SQL? Is there much overhead in the connection phase (thus: cache the connection) or can a developer just reconnect each time the database connection is required without much overhead? The target platform is extremely demanding in requests and memory, so I would like to design it good from the start. Stefan

2 1

[Monetdb-developers] MonetDB release planning
by Stefan Manegold 25 May '08

25 May '08

Dear fellow MonetDB developers, this is to keep you up-to-date about the short-term release plans for the MonetDB product family as they have recently been discussed here at CWI in A'dam. Please read until the end. Thank you very much in advance for your patience ans cooperation! Triggered by (1) the major progress in extending the functionality of the new "Algebra back-end" based version of Pathfinder (MonetDB/XQuery) to (almost) meet the functionality as provided by the (til date) default "milprint_summer back-end" based version (see details below), and (2) the desire of at least one distinct client to use the new "Algebra back-end" based version of Pathfinder (MonetDB/XQuery), we plan to have a new release of Pathfinder (MonetDB/XQuery), soon. Unfortunately, the functionality of the new "Algebra back-end" based version of Pathfinder (MonetDB/XQuery) is not yet identical with that of the (til date) default "milprint_summer back-end" based version. First of all, extensions beyond the XQuery and XQuery Updates standards, such as XRpc, StandOff, and pre-compiled XQuery modules ("MIL modules") are not supported, as well as recursive user-defined functions are not supported, yet. (The not yet working id/idref functionality is currently being implemented.) Moreover, the XQuery Update functionality is not yet complete/correct, as it cannot yet handle inserts of node who's type is not statically known at query compile time. In particular the latter shortcoming currently keeps us from our original plans of making the new "Algebra back-end" the default in the upcoming release. Hence, most probably, the upcoming release will still come with the "milprint_summer back-end" set as default. However, as already in the latest release, runtime options allow users to easily enable the new "Algebra back-end". Some (internal) API changes on the development trunk since the last release (mainly the signatures of GDK functions BAT_select(), BATmmap() & BATmadvice()), prohibit a new "feature release" of pathfinder, only, as this is incompatible with currently available releases of MonetDB4 (Server) and MonetDB (Common). Hence, also this upcoming release will be a "complete" one, i.e., all MonetDB related packages (MonetDB Common, Clients, MonetDB4 Server, MonetDB5 Server, MonetDB/SQL, MonetDB/Geom & MonetDB/XQuery will (have to be) released. Given this, our plan is as follows: Within the next week, we will fork new "Stable" branches of the current development trunks. Please let us know at your earliest convenience, in case you have new feature developments "in the pipe", that should go into the new release. In case you have just started new "cutting edge" developments that are still far from mature to be released soon, please refrain from checking them in until the new release branches have been created. Once the new release branches are created, we plan to spend about one month on thoroughly testing and stabilizing the new release and its packaging. Any help from all of you is highly appreciated! In particular, we count on the support for all "first users" of the new Pathfinder (MonetDB/XQuery) "Algebra back-end' (even if it will not be the default choice). The envisioned release date is "End of June". Please let us know, in case you have any questions, comments, suggestions, etc. Yours Sincerely, Stefan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

4 20

[Monetdb-developers] Independant SQL engine?
by Sylvain Bertrand 22 May '08

22 May '08

Hi, I would like to know if your SQL engine is independant enough from MonetDB in order to let me reasonably write my own SQL engine storage backend? (Not I dislike MonetDB, just that for the moment I use a custom disk storage and would try to put a SQL engine on top of it) Best regards, Sylvain B. -- use single GPL licensed software, use Linux and secure your digital freedom!

2 1

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/modules/pftijah serialize_pftijah.mx, , 1.60, 1.61
by Ying Zhang 15 May '08

15 May '08

Hi Jan, Your checkin breaks the compilation due to the following error: $SRCBASE_DIR/pathfinder/modules/pftijah/serialize_pftijah.mx: In function 'dbat_extend': $SRCBASE_DIR/pathfinder/modules/pftijah/serialize_pftijah.mx:175: error: too few arguments to function 'BATmmap' BATmmap has recently got an extra argument (please see Niels' checkin yesterday), so you might want to update your MonetDB modules firstly :) Regards, Jennie On May 15, 2008, at 11:59 , Jan Flokstra wrote: > Update of /cvsroot/monetdb/pathfinder/modules/pftijah > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv12778 > > Modified Files: > serialize_pftijah.mx > Log Message: > + repair pftijah direct memory acces problem which caused Henning's > vitalis > collection problems. > > > > U serialize_pftijah.mx > Index: serialize_pftijah.mx > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/modules/pftijah/ > serialize_pftijah.mx,v > retrieving revision 1.60 > retrieving revision 1.61 > diff -u -d -r1.60 -r1.61 > --- serialize_pftijah.mx 17 Apr 2008 10:29:10 -0000 1.60 > +++ serialize_pftijah.mx 15 May 2008 09:59:43 -0000 1.61 > @@ -135,9 +135,22 @@ > #define MINCHUNK 8192 > #define MAXCHUNK 67108864 > > -int dbat_extend(dbat* dbat, oid min_i /*raw-index*/, size_t > forced_size) { > +static void > +dbat_setcount(dbat *dbat, oid n) > +{ > + BAT *b = dbat->bat; > + BATsetcount(b, n); > + if (b->ttype) { > + b->tsorted = 0; > + b->tdense = 0; > + } > +} > + > +static int > +dbat_extend(dbat* dbat, oid min_i /*raw-index*/, size_t > forced_size) { > size_t newsize; > > + dbat_setcount(dbat, dbat->raw_max); > if ( forced_size ) { > newsize = forced_size; > } else { > @@ -156,10 +169,19 @@ > return -1; > } > } > +#if 1 > + /* copied this form shredder.mx, keep it here ??? */ > + if (newsize+newsize > (size_t) REMAP_PAGE_MAXSIZE) { /* try to > use mmap() */ > + BATmmap(dbat->bat, STORE_MMAP, STORE_MMAP, STORE_MMAP); > + } > + > +#endif > + > #if 0 > stream_printf(GDKout,"dbat_extend[%s](%d -> %d)\n",dbat- > >name,dbat->raw_max,newsize); > #endif > - dbat->raw_max= newsize; > + > + dbat->raw_max = newsize; > dbat->seq_max = dbat->raw_max + dbat->seqbase; > if ( !(dbat->bat = BATextend(dbat->bat,newsize)) ) { > GDKerror("dbat_extend: BATextend[\"%s\"](to %d) fails > \n","incomplete",newsize); > @@ -169,17 +191,30 @@ > return 1; > } > > -int dbat_sizeHint(dbat* dbat, int sizeHint_mark) { > +#if 0 > +static int > +dbat_sizeHint(dbat* dbat, int sizeHint_mark) { > int sizeHint = sizeHint_mark - dbat->seqbase; > int estimate = dbat->raw_max + sizeHint; > > return dbat_extend(dbat, estimate, 0); > } > +#endif > > #define dbat_set_oid(DBAT,I,V) (DBAT)->cast.oidCAST[I-(DBAT)- > >seqbase] = V > > #define dbat_set_int(DBAT,I,V) (DBAT)->cast.intCAST[I-(DBAT)- > >seqbase] = V > > +#if 0 > +static void > +dbat_int_debug(char* t, dbat* dbat) { > + stream_printf(GDKout,"+ DEBUGGING INT DBAT[sz=%d]@=[%s]:\n",dbat- > >seq_max,t); > + for(int i=0; i<10; i++) { > + stream_printf(GDKout,"+ dbat[%d] = %d\n",i,dbat- > >cast.intCAST[i]); > + } > +} > +#endif > + > /************************************************ > * > * The Tijah-Pathfinder index creation context > @@ -391,7 +426,7 @@ > if ( (tjPre = tj_newPre(tjctx) ) == oid_nil ) > return 0; > dbat_set_oid(&tjctx->dbat_collPre, tjPre, termOid); > - dbat_set_int(&tjctx->dbat_collSize,(int)tjPre,0); > + dbat_set_int(&tjctx->dbat_collSize,tjPre,0); > #ifdef TJ_TRACE > if ( TJ_TRACE ) stream_printf(GDKout,"C[%s]:TERM: \"%s > \", termoid=%d, tjPre=%d\n",tjctx->name,term,termOid,tjPre); > #endif > @@ -1230,7 +1265,7 @@ > return GDK_FAIL; > if ( dbat_finalize(&tjctx->dbat_collSize, tjctx->tijahPre) < 0 ) > return GDK_FAIL; > - > + > tjctx->b_collPre->batDirty = TRUE; > tjctx->b_collSize->batDirty = TRUE; > tjctx->b_collPfPre->batDirty = TRUE; > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins

1 0