- developers-list - monetdb.org

Re: [Monetdb-developers] [Monetdb-sql-checkins] sql/src/test/leaks/Tests check0.stable.out, 1.1.2.2, 1.1.2.3 check1.stable.out, 1.1.2.2, 1.1.2.3 check2.stable.out, 1.1.2.2, 1.1.2.3 check3.stable.out, 1.1.2.2, 1.1.2.3 check4.stable.out, 1.1.2.1, 1.1.2.2 check5.stable.out, 1.1.2.1, 1.1.2.2
by Martin Kersten 13 Dec '06

13 Dec '06

ziek ?:-) of alleen teveeel gedanst.

2 1

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/runtime pathfinder.mx,
by p.a.boncz＠chello.nl 13 Dec '06

13 Dec '06

Hoi Sjoerd, In de log message ben ik denk ik vergeten om op te schrijven dat ik consecutive text node coalescing uit had gezegd; daarentegen heb ik je een uitgebreide email geschreven waarin ik dit verteld heb dat en waarom ik dat gedaan heb. De fix_consecutive_text code heeft een groot performance probleem. Het maakt in feite deletes onbruikbaar. Op de 1GB document test heb ik hem na enige minuten afgebroken. Omdat ik niet wist hoe die code sneller gemaakt kan worden heb ik het uitgezet. Hoewel technisch niet correct, is het voor weinig applicaties een ramp als er twee text nodes na elkaar komen. De pathfinder gegenereerde code breekt daar niet van, voor zover ik weet. Maar ik begrijp dat jij het hier niet mee eens bent.. Het beste zou zijn om fix-consecutive text te beperken tot de range van nodes direct om het delete punt heen. Dan zou performance geen probleem hoeven te zijn. Wat ik zag was dat er een CTrefine over het hele document heen ging. Peter > > van: Sjoerd Mullender <sjoerd(a)acm.org> > datum: 2006/12/12 Tue PM 04:11:49 CET > aan: monetdb-developers(a)lists.sourceforge.net > cc: monetdb-pf-checkins(a)lists.sourceforge.net > onderwerp: Re: [Monetdb-pf-checkins] pathfinder/runtime pathfinder.mx, > 1.269.2.14, 1.269.2.15 pf_support.mx, 1.172.2.11, 1.172.2.12 > >

2 1

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/runtime pathfinder.mx, 1.269.2.14, 1.269.2.15 pf_support.mx, 1.172.2.11, 1.172.2.12
by Sjoerd Mullender 12 Dec '06

12 Dec '06

On 2006-12-09 16:19, Peter Boncz wrote: > Update of /cvsroot/monetdb/pathfinder/runtime > In directory sc8-pr-cvs7.sourceforge.net:/tmp/cvs-serv28149 > > Modified Files: > Tag: XQuery_0-14 > pathfinder.mx pf_support.mx > Log Message: > various performance fixes; small inserts and deletes are now reasonably fast > > - index new qnames in qnloc index > - better handling of the hash tables on the qn_loc index > - shortened the collection lock time by moving most of the logging > out of the critical period > > > > Index: pf_support.mx > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/runtime/pf_support.mx,v > retrieving revision 1.172.2.11 > retrieving revision 1.172.2.12 > diff -u -d -r1.172.2.11 -r1.172.2.12 > --- pf_support.mx 8 Dec 2006 17:48:43 -0000 1.172.2.11 > +++ pf_support.mx 9 Dec 2006 15:19:52 -0000 1.172.2.12 > @@ -2096,47 +2097,22 @@ > } > > var texts := ws.fetch(UPDATED_TEXT); > - fix_consecutive_texts(ws, texts); > + #fix_consecutive_texts(ws, texts); > > # get containers in the order in which we must commit them ([COLL_ID,CONT]) > var cont_order := affected_conts.reverse().mirror().leftfetchjoin(ws.fetch(CONT_COLL)).reverse().sort(); > 1. what does this change have to do with the log message? 2. this change breaks the document: if a node between two text nodes is deleted, the two text nodes have to be coalesced. That is what this function does. It is a necessary step to keep subsequent queries working correctly. -- Sjoerd Mullender

1 0

[Monetdb-developers] MonetDB as MySQL storage engine
by Arman Bostani 12 Dec '06

12 Dec '06

Hello all, Has anyone considered the possibility of using MonetDB as a MySQL storage engine. This idea is appealing since, theoretically, you'd immediately have access to MySQL features such as clustering, replication, etc. I would guess, however, that you would lose much of MonetDB performance advantages. Any thoughts? -arman

2 1

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder MonetDB-XQuery.spec.in, 1.16, 1.17 Makefile.ag, 1.22, 1.23 extra.mk, 1.18, 1.19
by Stefan Manegold 09 Dec '06

09 Dec '06

Niels, these changes are not meant for the release? Stefan On Sat, Dec 09, 2006 at 11:13:52AM +0000, Niels Nes wrote: > Update of /cvsroot/monetdb/pathfinder > In directory sc8-pr-cvs7.sourceforge.net:/tmp/cvs-serv22364 > > Modified Files: > MonetDB-XQuery.spec.in Makefile.ag extra.mk > Log Message: > install howtos is done within the monetweb tools > > > Index: Makefile.ag > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/Makefile.ag,v > retrieving revision 1.22 > retrieving revision 1.23 > diff -u -d -r1.22 -r1.23 > --- Makefile.ag 30 Nov 2006 11:56:57 -0000 1.22 > +++ Makefile.ag 9 Dec 2006 11:13:50 -0000 1.23 > @@ -26,11 +26,21 @@ > > EXTRA_DIST = rules.mk extra.mk RunMtest.in RunMtest.bat.in \ > RunMapprove.in RunMapprove.bat.in \ > - MonetDB-XQuery.spec.in MonetDB-XQuery.spec rpm.mk.in \ > - HowToStart-PF HowToStart-MinGW.txt > + MonetDB-XQuery.spec.in MonetDB-XQuery.spec rpm.mk.in > EXTRA_DIST_DIR = NT tests > > EXTRA_HEADERS = pf_config.h > > +scripts_txt = { > + SOURCES = HowToStart-MinGW.txt > + DIR = datadir/doc/MonetDB-XQuery-${VERSION} > +} > + > +scripts_docs = { > + EXT = "" > + SOURCES = HowToStart-PF > + DIR = datadir/doc/MonetDB-XQuery-${VERSION} > +} > + > TESTS = RunMtest > TESTS_ENVIRONMENT = $(SHELL) > > Index: extra.mk > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/extra.mk,v > retrieving revision 1.18 > retrieving revision 1.19 > diff -u -d -r1.18 -r1.19 > --- extra.mk 8 Nov 2006 13:16:44 -0000 1.18 > +++ extra.mk 9 Dec 2006 11:13:50 -0000 1.19 > @@ -68,19 +68,3 @@ > # > doc: > cd $(top_srcdir) ; doc/gen_doc.sh > - > -# > -# Stefan's automated test system uses `make docs', not `make doc'. > -# Make him happy as well. > -# > -docs: \ > - $(prefix)/doc/MonetDB/GetGoing/Setup/XQuery/Unix/index.html \ > - $(prefix)/doc/MonetDB/GetGoing/Setup/XQuery/Windows/index.html > - > -$(prefix)/doc/MonetDB/GetGoing/Setup/XQuery/Unix/index.html: $(top_srcdir)/HowToStart-PF > - mkdir -p $(prefix)/doc/MonetDB/GetGoing/Setup/XQuery/Unix > - rst2html.py --stylesheet http://monetdb.cwi.nl/MonetDB.css --link-stylesheet $(top_srcdir)/HowToStart-PF > $(prefix)/doc/MonetDB/GetGoing/Setup/XQuery/Unix/index.html > - > -$(prefix)/doc/MonetDB/GetGoing/Setup/XQuery/Windows/index.html: $(top_srcdir)/HowToStart-MinGW.txt > - mkdir -p $(prefix)/doc/MonetDB/GetGoing/Setup/XQuery/Windows > - rst2html.py --stylesheet http://monetdb.cwi.nl/MonetDB.css --link-stylesheet $(top_srcdir)/HowToStart-MinGW.txt > $(prefix)/doc/MonetDB/GetGoing/Setup/XQuery/Windows/index.html > > Index: MonetDB-XQuery.spec.in > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/MonetDB-XQuery.spec.in,v > retrieving revision 1.16 > retrieving revision 1.17 > diff -u -d -r1.16 -r1.17 > --- MonetDB-XQuery.spec.in 1 Nov 2006 17:48:37 -0000 1.16 > +++ MonetDB-XQuery.spec.in 9 Dec 2006 11:13:50 -0000 1.17 > @@ -21,8 +21,6 @@ > > %define comp_cc @CC@ > > -%define builddoc 0 > - > # groups of related archs > %define all_x86 i386 i586 i686 > > @@ -47,15 +45,6 @@ > Requires: %{name}-server > Requires: MonetDB-devel > > -%if %{builddoc} > -%package docu > -Summary: MonetDB XQuery documentation package > -Group: System > - > -%description docu > -Add the MonetDB documentaion description here > -%endif > - > %prep > rm -rf $RPM_BUILD_ROOT > > @@ -95,18 +84,8 @@ > %{_libdir}/MonetDB/*.mil > %{prefix}/bin/Mbeddedxq > %{_libdir}/libembeddedxq.* > +%{_datadir}/doc/%{name}-%{version}/* > > %files devel > %defattr(-,monetdb,monetdb) > %{prefix}/include/pathfinder/*.h > - > -%if %{builddoc} > -%files docu > -%defattr(-,monetdb,monetdb) > -%{prefix}/share/MonetDB/docs/pathfinder/*.html > -%{prefix}/share/MonetDB/docs/pathfinder/*.pdf > -%{prefix}/share/MonetDB/docs/pathfinder/*.ps > -%{prefix}/share/MonetDB/docs/pathfinder/*/*.html > -%{prefix}/share/MonetDB/docs/pathfinder/*/*.pdf > -%{prefix}/share/MonetDB/docs/pathfinder/*/*.ps > -%endif > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys - and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 1

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB/doc mxdoc.tex, 1.5, NONE
by Stefan Manegold 08 Dec '06

08 Dec '06

Martin, just two observations I made when seein these checkins: - the changes were made on the development trunk, and hence will not be in the release. - `make docs` for MonetDB will fail, now as MonetDB/html.mk still refers to the no longer existing MonetDB/doc/mxdoc.tex Stefan (yes, I am German and will always be!) On Fri, Dec 08, 2006 at 09:08:48AM +0000, Martin Kersten wrote: > Update of /cvsroot/monetdb/MonetDB/doc > In directory sc8-pr-cvs7.sourceforge.net:/tmp/cvs-serv16022 > > Removed Files: > mxdoc.tex > Log Message: > Mx documentation is moved to the buildtools. > > > --- mxdoc.tex DELETED --- > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys - and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 1

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/compiler/mil milprint_summer.c, 1.298.2.4, 1.298.2.5
by Stefan Manegold 07 Dec '06

07 Dec '06

Peter, the changes below seem to make 3 of the 6 Music/limit tests fail with " !ERROR: interpret: no matching MIL operator to 'seqbase(int, oid)'. !MAYBE YOU MEAN: ! seqbase(BAT[void,any::1], oid) : BAT[void,any::1] ! seqbase(BAT[void,any::1]) : oid ! seqbase(BAT[oid,any::1]) : oid " What kind of solution/fix do you recommend? (1) set the seqbase only if the type is BAT, or (2) add a "fake" constant-seqbase variant to the constant module ? Thanks! Stefan On Tue, Nov 28, 2006 at 02:18:09PM +0000, Peter Boncz wrote: > Update of /cvsroot/monetdb/pathfinder/compiler/mil > In directory sc8-pr-cvs7.sourceforge.net:/tmp/cvs-serv14236 > > Modified Files: > Tag: XQuery_0-14 > milprint_summer.c > Log Message: > - [ 1600704 ] XQuery: subsequence > 1 fails > > we require the seqbase of our iter|pos|item relations to be 0@0 > > > > Index: milprint_summer.c > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/compiler/mil/milprint_summer.c,v > retrieving revision 1.298.2.4 > retrieving revision 1.298.2.5 > diff -u -d -r1.298.2.4 -r1.298.2.5 > --- milprint_summer.c 16 Nov 2006 17:52:56 -0000 1.298.2.4 > +++ milprint_summer.c 28 Nov 2006 14:18:06 -0000 1.298.2.5 > @@ -7641,10 +7641,10 @@ > > milprintf(f, "\n" > " # select a slice\n" > - " ipik := ipik.slice(int(lo),hi);\n" > - " iter := iter.slice(int(lo),hi);\n" > - " kind := kind.slice(int(lo),hi);\n" > - " item%s := item%s.slice(int(lo), hi);\n" > + " ipik := ipik.slice(int(lo),hi).seqbase(0@0);\n" > + " iter := iter.slice(int(lo),hi).seqbase(0@0);\n" > + " kind := kind.slice(int(lo),hi).seqbase(0@0);\n" > + " item%s := item%s.slice(int(lo), hi).seqbase(0@0);\n" > " pos := ipik.mark(1@0);\n" > "} else {\n" > " # evaluate selection tuple by tuple (note: fully constant-resistant code)\n" > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys - and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 1

[Monetdb-developers] Compiling M4 release...
by Romulo Goncalves 07 Dec '06

07 Dec '06

Dear developers I was updating my M4 release and I got the following error: gcc -m64 -DHAVE_CONFIG_H -I. -I/ufs/goncalve/MonetDB/MonetDB_Release/MonetDB4/MonetDB/src/modules/plain -I../../.. -I/ufs/goncalve/MonetDB/MonetDB_Release/MonetDB4/MonetDB/src/modules/plain -I../../common -I/ufs/goncalve/MonetDB/MonetDB_Release/MonetDB4/MonetDB/src/modules/plain/../../common -I../../gdk -I/ufs/goncalve/MonetDB/MonetDB_Release/MonetDB4/MonetDB/src/modules/plain/../../gdk -I../../monet -I/ufs/goncalve/MonetDB/MonetDB_Release/MonetDB4/MonetDB/src/modules/plain/../../monet -DLIBUPGRADE -Wall -W -std=c99 -g -Werror-implicit-function-declaration -Werror -Wno-unused-function -Wno-unused-label -D_REENTRANT -c upgrade.c -fPIC -DPIC -o .libs/lib_upgrade_la-upgrade.o /ufs/goncalve/MonetDB/MonetDB_Release/MonetDB4/MonetDB/src/modules/plain/upgrade.mx: In function 'bat_convert': /ufs/goncalve/MonetDB/MonetDB_Release/MonetDB4/MonetDB/src/modules/plain/upgrade.mx:465: error: 'Heap' has no member named 'padding' /ufs/goncalve/MonetDB/MonetDB_Release/MonetDB4/MonetDB/src/modules/plain/upgrade.mx:486: error: 'Heap' has no member named 'padding' /ufs/goncalve/MonetDB/MonetDB_Release/MonetDB4/MonetDB/src/modules/plain/upgrade.mx:509: error: 'Heap' has no member named 'padding' make[7]: *** [lib_upgrade_la-upgrade.lo] Error 1 make[7]: Leaving directory `/net/amelia/export/scratch1/goncalve/MonetDB/MonetDB_Release/monetdb4/build/src/modules/plain' make[6]: *** [all] Error 2 I also updated the buildtools. It should a be a stable branch, right? ;) Regards, Romulo

2 1

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/compiler/semantics normalize.brg, 1.11, 1.11.2.1
by Jan Rittinger 06 Dec '06

06 Dec '06

On 12/01/2006 02:28 AM, Peter Boncz wrote with possible deletions: > The Moerkotte Queries (e.g. /ufs/boncz/moer) seduced me into implementing > the "syntactic variant" solution to the problem discussed by Jens in the wiki: > [...] > I do *not* propose to propagate this to the HEAD automatically. The decision whether > this rewrite is worth the while I leave to the Munich folks. It is a thing that helps > mps (a lot, in fact for programmers that use X[Z] XPath notation rather than xquery > notation for $x in X where Z return $x) > Stefan's automatic propagation however already did exactly that... (I just played around with both versions and it seems that the algebra is not slower with these modifications. In Guidos q11 the modified version is even 500 msec faster. My interpretation is that some more rewrites on the algebra side are needed to fill the gap :).) > Anyway, on our standard benchmarking machine db2 (non-optimized 64-bits 32-bits oids > gcc compile) we now get the following score on the Moerkotte Queries: > > MXQ.14 MXQ.12 saxon > q0 0.67 1.71 1.5 > q1 0.26 0.33 1.6 > q2 0.68 1.23 60 > q3 2.38 DNF 209 > q4 1.83 2.02 61 > q5 2.74 DNF 166 > q6 4.82 DNF 130 > q7 2312 DNF 211 > q8 DNF DNF DNF > q9 DNF DNF DNF > q10 DNF DNF DNF > q11 DNF DNF 151 > q12 0.14 DNF ? I tried to get the numbers out of the algebra branch as well. On my Laptop (1700MHz, 1024MB Ram) I got better numbers for the first two queries compiled with milprint_summer (I only tried these two). The algebra version compiles for 7 of the 13 queries (3 fail due to non existent reverse sorting in the algebra and the 3 others fail as f n:number is not implemented yet). Queries q0, q1, q2, q4, q5, q11, and q12 compile and in q0, q1, q5, q11, and q12 all joins are recognized (at least as far as I could see). q2 and q4 still each contain an undetected join because we lack the detection of conjunctive joins (until now). In the following the measurements of a single(!) run for each query: guido q0: alg: Comp 613.000 msec Load 51.719 msec Query 224.477 msec Print 1.319 msec mps: Trans 242.385 msec Shred 59.625 msec Query 385.546 msec Print 1.128 msec guido q1: alg: Comp 79.000 msec Load 52.124 msec Query 120.840 msec Print 16.844 msec mps: Trans 62.809 msec Shred 51.907 msec Query 155.992 msec Print 17.736 msec guido q2 - alg: Comp 473.000 msec DNF (dependent join) guido q4 - alg: Comp 522.000 msec DNF (dependent join) guido q5 - alg: Comp 03s 255 msec Load 50.975 msec Query 1369.837 msec Print 158.239 msec guido q11 - alg: Comp 02s 010 msec Load 50.960 msec Query 1130.238 msec Print 74.269 msec guido q12 - alg: Comp 145.000 msec Load 52.192 msec Query 53.572 msec Print 1.980 msec -- Jan Rittinger Database Systems Technische Universität München (Germany) http://www-db.in.tum.de/~rittinge/

3 2

[Monetdb-developers] bootstrap: why does it remove .orig files?
by Fabian Groffen 06 Dec '06

06 Dec '06

Hi all, I just found out I lost some (hours of) work, because de-bootstrap threw away some files which I temporarily renamed to "*.orig". Whether or not this was a good idea, I'm wondering what de-bootstrap's rationale is to remove those "original" files, which I don't think that match the description of de-bootstrap: This script ("./de-bootstrap") can be used to clean up the source tree, i.e., to remove all files that were created by "./bootstrap". <rant> In my case it did remove a file that was NOT created by bootstrap, and if bootstrap ever has a patch failing and produces an .orig (and .rej) file, I might prefer my unclobbered file to exist. In any case, I am NOT amused, and think -- considering the many wildcards in the script -- that it does far from what its description says. Of course it asks me whether or not I want to delete stuff, but sorry, I haven't written a script yet that performs the task de-bootstrap should have done to see whether it throws away a file it shouldn't even want to touch. And manually examining the two terminal heights long list of files also isn't my speciality. </rant>

2 2