- developers-list - monetdb.org

[Monetdb-developers] Announcement: New Aug2009-SP1 bugfix release of MonetDB suite
by Sjoerd Mullender 28 Sep '09

28 Sep '09

The MonetDB team at CWI/MonetDB BV is pleased to announce the Aug2009-SP1 bug fix release of the MonetDB suite of programs. This is the first bug fix release for the Aug2009 release. More information on this release can be found at <http://monetdb.cwi.nl/projects/monetdb//Development/Releases/Aug2009/#Aug20…>. The sources and binaries are available at <http://monetdb.cwi.nl/projects/monetdb/Download/>. -- Sjoerd Mullender

1 0

[Monetdb-developers] Remote Merovingian control
by Fabian Groffen 25 Sep '09

25 Sep '09

This message only applies to the current development branch of MonetDB, and is informational only, mainly addressed to others working with the current development branch. monetdb[1] goes abroad After quite some hacking to move code back and forth between components, we now got the first version of merovingian[2] and monetdb[1] that can do remote control. I will give some small examples here so everyone that is interested can get up to speed fast, and/or get an idea on the possibilities. Basically, all functionality that used to be implemented in monetdb, is now implemented in merovingian. That means, for instance creation of a database is performed by merovingian. A positive side-effect of this is that the discovery aspect of databases is now real-time, e.g.: % monetdb release dumptest taken database out of maintenance mode: dumptest 2009-09-25 11:54:10 MSG control[8590]: (local): released database 'dumptest' 2009-09-25 11:54:10 MSG discovery[8590]: new database mapi:monetdb://pegasus.ins.cwi.nl:50000/dumptest (ttl=660s) We can see that when database is released again using monetdb release, it is immediately announced in the network, making it available for use. Locking a database has the oposite effect, of course. Next, the "(local)" string indicates that this control command was received over the local UNIX socket, as was the case in previous releases of merovingian. In addition to this local access, monetdb now also can connect over a TCP socket, which needs not to be remote, e.g.: % monetdb -hjarl.ins.cwi.nl -Pxxxxxx status name state uptime health last crash testdbatjenniesmachine stopped 100%, 4m - 2009-09-25 11:56:19 MSG control[11449]: pegasus.ins.cwi.nl:49652: served status list Here we can see that the merovingian at jarl reports it served a status list to pegasus, which is the host I performed the monetdb status at. Note the -Pxxxxxxxx in there, which is the passphrase of the remote merovingian. This passphrase is by default a random string, but can be set to something else by editing the .merovingian_pass file in the dbfarm and restarting merovingian. The way to find other merovingians in the vincinity is by using monetdb discover. Its output now includes how to reach each merovingian, if remote control is enabled for them. Remote control over TCP can be disabled in monetdb5.conf by unsetting mero_controlport. Control entries can be recognised in discover's output by the trailing asterisk, which one can also glob for: % monetdb discover "*\*" location pegasus.ins.cwi.nl:50001* volund.ins.cwi.nl:50001* jarl.ins.cwi.nl:64001* This command just returned all merovingians that have remote control enabled and reports their hostname and port to connect to. The remote control protocol is very simple, and I'm planning to make at least a Java library available to perform these commands, such that monetdb[1] no longer needs to be the only application to perform commands with merovingian. [1] http://homepages.cwi.nl/~fabian/MonetDB/Man%20Pages.html/monetdb.html [2] http://homepages.cwi.nl/~fabian/MonetDB/Man%20Pages.html/merovingian.html

1 0

[Monetdb-developers] Embedded MonetDb under Windows
by Andzej Tatul 22 Sep '09

22 Sep '09

Hello I have installed MonedDB Sql (32 bit) under Windows (xp), but cannot run it as embedded server. In MonetDB documentation it is mentioned that embedded monetDb engine is available. How can I get embedded server installation for windows? Best regards Andzej _________________________________________________________________ Daugiau nei pranešimai – išbandykite likusias „Windows Live™“ dalis. http://www.microsoft.com/windows/windowslive/

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/modules/mal/crackers crackers.mx, , 1.16, 1.17 crackers_sideways.mx, , 1.21, 1.22
by Stefan Manegold 16 Sep '09

16 Sep '09

Stratos (and others ;-)), please find below some comments, hints and questions. Stefan On Wed, Sep 16, 2009 at 09:53:56PM +0000, stratos wrote: > Update of /cvsroot/monetdb/MonetDB5/src/modules/mal/crackers > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv11656/src/modules/mal/crackers > > Modified Files: > crackers.mx crackers_sideways.mx > Log Message: > position-based tuple reconstruction to use in sideways plans with crack joins > > > U crackers.mx > Index: crackers.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/modules/mal/crackers/crackers.mx,v > retrieving revision 1.16 > retrieving revision 1.17 > diff -u -d -r1.16 -r1.17 > --- crackers.mx 30 Apr 2009 18:43:40 -0000 1.16 > +++ crackers.mx 16 Sep 2009 21:53:53 -0000 1.17 > @@ -504,6 +504,10 @@ > address CRKdProject_@1_@2 > comment "Sync the cracking pair and project the tail. Use for disjunctive queries that require a larger bit vector"; > > +command positionproject( b:bat[:oid,:@1],bp:bat[:oid,:@2],pivot:bat[:oid,:oid]):bat[:oid,:@2] > +address CRKdPositionProject_@1_@2 > +comment "Sync the cracking pair and project the tail. Use position based pivot"; > + > command projectH( b:bat[:oid,:@1],bp:bat[:oid,:@2],l:@1,h:@1,li:bit,hi:bit):bat[:oid,:@1] > address CRKProjectH_@1_@2 > comment "Sync the cracking pair and project the head"; > > U crackers_sideways.mx > Index: crackers_sideways.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/modules/mal/crackers/crackers_sideways.mx,v > retrieving revision 1.21 > retrieving revision 1.22 > diff -u -d -r1.21 -r1.22 > --- crackers_sideways.mx 30 Apr 2009 18:43:40 -0000 1.21 > +++ crackers_sideways.mx 16 Sep 2009 21:53:53 -0000 1.22 > @@ -153,6 +153,7 @@ > crackers_export str CRKMarkedProject_@1_@2(int *res, int *bid, int *pbid, int *pivot, @1 *low, @1 *hgh, bit *inclusiveLow, bit *inclusiveHgh); > crackers_export str CRKPositionProject_@1_@2(int *res, int *bid, int *pbid, int *PositionPivot, @1 *low, @1 *hgh, bit *inclusiveLow, bit *inclusiveHgh); > crackers_export str CRKdProject_@1_@2(int *res, int *bid, int *pbid, int *pivot); > +crackers_export str CRKdPositionProject_@1_@2(int *res, int *bid, int *pbid, int *pivot); > @ > > @{ > @@ -1025,6 +1026,63 @@ > > return MAL_SUCCEED; > } > + > +str > +CRKdPositionProject_@1_@2(int *res, int *bid, int *pbid,int *pivot){ > + BAT *b, *p, *r, *c; > + @2 *pairt, *rest; > + oid *pivott; > + oid size; > + int m; > + oid j; > + > + /* If the necessary map does not exits, then create it */ > + m = existsCrackerIndexSideways(*bid, *pbid); > + if (m==-1){ > + @:CreateCrackerMap(@1,@2)@ > + BBPkeepref(b->batCacheid); > + BBPkeepref(c->batCacheid); > + } > + > + /* Align the map based on the cracker tape of this set */ > + Sync_@1_@2(m,existsCrackerIndexSideways(*bid, -1)); > + > + /* Get the map */ > + if ((b = BATdescriptor(CrackerIndex[m].cbid)) == NULL) > + throw(MAL, "crackers.CRKProject", "Cannot access descriptor"); > + > + if ((p = BATdescriptor(*pivot)) == NULL) > + throw(MAL, "crackers.CRKProject", "Cannot access descriptor"); > + > + size = BATcount(p); > + r = BATnew(TYPE_void,b->ttype,size); This might fail. Error handling? > + > + pairt = (@2 *)Tloc(b, BUNfirst(b)); > + pivott = (oid*)Tloc(p, BUNfirst(p)); > + rest = (@2 *)Tloc(r, BUNfirst(r)); > + > + for (j = 0; j<size; j++){ > + *rest = *(pairt + *pivott); > + rest ++; > + pivott ++; What about replacing the above three lines by rest[j] = pairt[pivott[j]]; ? > + } > + > + BATmark(BATmirror(p),0); ^^^^^^^^^^^^^^^^^^!^^^^^ What's is the purpose/intention of this statement? It IMHO represents a side-effect, since it marks one of the input operands ... Stefan > + > + BATsetcount(r, size); > + > + r->hsorted = FALSE; > + r->tsorted = FALSE; > + r->hdense = FALSE; > + r->tdense = FALSE; > + > + *res = r->batCacheid; > + BBPkeepref(r->batCacheid); > + BBPunfix(p->batCacheid); > + BBPunfix(b->batCacheid); > + > + return MAL_SUCCEED; > +} > @ > > @- Local support functions and macros > > > ------------------------------------------------------------------------------ > Come build with us! The BlackBerry® Developer Conference in SF, CA > is the only developer event you need to attend this year. Jumpstart your > developing skills, take BlackBerry mobile applications to market and stay > ahead of the curve. Join us from November 9-12, 2009. Register now! > http://p.sf.net/sfu/devconf > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 1

[Monetdb-developers] monentdb autocommit
by 颜勋 07 Sep '09

07 Sep '09

Hi all, I use BenchmarkFactory and ODBC to test the monentdb performance. But it seems the default option of monentdb is autocommit, so some test scripts seems not work. How should i change the autocommit option (not only use "set autocommit 0" and just change it in the range of one transaction )? Does anyone know the details(that'are tools, scripts, monetdb enviroment) of TPC-H test in monentdb website? Many thanks~ Regards Yan

3 2

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/mal/Tests pithole.mal, Aug2009, NONE, 1.1.2.1 All, Aug2009, 1.113, 1.113.2.1
by Stefan Manegold 06 Sep '09

06 Sep '09

Martin, this checkin seems to contain only the new test. Did you by accident forget to checkin the related code changes? Stefan On Sun, Sep 06, 2009 at 07:40:28AM +0000, Martin Kersten wrote: > Update of /cvsroot/monetdb/MonetDB5/src/mal/Tests > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv29612 > > Modified Files: > Tag: Aug2009 > All > Added Files: > Tag: Aug2009 > pithole.mal > Log Message: > Defense against recursion segfaults. > > > --- NEW FILE: pithole.mal --- > #challenge the runtime stack protection > user.main(); > > U All > Index: All > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/mal/Tests/All,v > retrieving revision 1.113 > retrieving revision 1.113.2.1 > diff -u -d -r1.113 -r1.113.2.1 > --- All 28 Jun 2009 20:53:50 -0000 1.113 > +++ All 6 Sep 2009 07:40:23 -0000 1.113.2.1 > @@ -250,3 +250,4 @@ > flowchk > > dynamicload > +pithole.mal > > > ------------------------------------------------------------------------------ > Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day > trial. Simplify your report design, integration and deployment - and focus on > what you do best, core application coding. Discover what's new with > Crystal Reports now. http://p.sf.net/sfu/bobj-july > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 1

[Monetdb-developers] Think someone forgot something again...
by Stefan de Konink 04 Sep '09

04 Sep '09

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 !SyntaxException:slave.close[0]:RETURN missing Stefan -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEAREKAAYFAkqgR5MACgkQYH1+F2Rqwn2RSQCeKz0nIOwSoT51GTAA3w6nWHz1 WKgAn3uV7MmDc2vwQz0YALzwDf3AOYly =HqhO -----END PGP SIGNATURE-----

2 2

[Monetdb-developers] New login intentional?
by Stefan de Konink 04 Sep '09

04 Sep '09

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 It seems to be that there is a new mechanism that basically doesn't allow mclient to connect directly anymore. Is this behavior intentional? The workaround seems to be -umonetdb with monetdb as password. Stefan -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEAREKAAYFAkqgSCkACgkQYH1+F2Rqwn3bGgCeIIj438dKmOp1lasT7sGbgdgE Ow0An1BBzIVLAr7cuLRhsOcexCP287GY =qRmk -----END PGP SIGNATURE-----

2 1

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/modules/mal replicator.mx, , 1.14, 1.15
by Stefan de Konink 29 Aug '09

29 Aug '09

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 ../../../../src/modules/mal/replicator.mx: In function ‘MASTERreplay’: ../../../../src/modules/mal/replicator.mx:551: error: ‘msg’ may be used uninitialized in this function make[8]: *** [lib_replicator_la-replicator.lo] Error 1 I get the following nag from the compiler. Stefan -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEAREKAAYFAkqZWRYACgkQYH1+F2Rqwn2GYACcCyJVoQQzXV72lthLqcuzzCGx ef0AniCCaJy8qHd0IZYjeccYOFwvRCgo =xUjB -----END PGP SIGNATURE-----

2 1

[Monetdb-developers] MonetDB/XQuery: reading XML files from TAR archives
by Wouter Alink 27 Aug '09

27 Aug '09

Hello devs, Roberto and I yesterday discussed that it would be useful to be able to load (compressed) XML collections directly into MonetDB/XQuery. The attached diff provides a new feature for loading multiple XML docs directly from tar files. Usage: "mclient -lxq -C <colname>" and pass a tarfile via stdin, see example below. My question: is this useful enough to make it into MonetDB? And if so, is the current syntax appropriate. Comments are appreciated. Greetings, Wouter $ mkdir xmlfiles $ echo "<abc/>" > xmlfiles/aap.xml $ echo "<abc/>" > xmlfiles/beer.xml $ tar cf xmlfiles.tar xmlfiles $ mclient -lxq -C xmlfiles < xmlfiles.tar Copying TAR file into collection: 'xmlfiles' Name: xmlfiles/beer.xml Length: 7 Name: xmlfiles/aap.xml Length: 7 $ echo 'pf:documents("xmlfiles")' | mclient -lxq <document updatable="false" url="xmlfiles/aap.xml" collection="xmlfiles">xmlfiles/aap.xml</document>, <document updatable="false" url="xmlfiles/beer.xml" collection="xmlfiles">xmlfiles/beer.xml</document> $

2 2