- developers-list - monetdb.org

[Monetdb-developers] Mapi wire protocol level 9
by Fabian.Groffen＠cwi.nl 11 May '09

11 May '09

Hi, As I'm working on changing the password backend of M5 to use hashes instead of cypher3d clear text passwords[1], I have to bump the wire protocol level from 8 to 9, since the client will have to construct the challenge response differently. Since this is a bump, this is a good time to also incorporate (or prepara) any other changes that we would like to see in the wire protocol for the next feature release. [1] to allow sql to dump users and passwords in a safe manner, the dump needs to be able restore passwords from a hash, which are then stored in the backend. Hence the backend needs to work with those hashes instead of clear text passwords, as it currently does.

2 1

Re: [Monetdb-developers] [Monetdb-sql-checkins] sql/src/backends/monet5 sql_user.mx, , 1.42, 1.43
by Stefan de Konink 11 May '09

11 May '09

Fabian wrote: > int len = strlen(passwd); > - if ((ret = AUTHSHA1Sum(&pwd, &passwd, &len)) != MAL_SUCCEED) { > + if ((ret = AUTHBackendSum(&pwd, &passwd, &len)) != MAL_SUCCEED) { A sum of 0 will not work. Stefan

2 1

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/compiler/include Makefile.ag, , 1.52, 1.53 logdebug.h, , 1.9, 1.10 properties.h, , 1.51, 1.52 xml2lalg.h, , 1.6, 1.7
by Stefan Manegold 07 May '09

07 May '09

Jan, could you please also check in the below included header file "plan_bundle.h" ? Thanks! Stefan On Thu, May 07, 2009 at 08:00:19PM +0000, Jan Rittinger wrote: > Update of /cvsroot/monetdb/pathfinder/compiler/include > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv23317/include > > Modified Files: > Makefile.ag logdebug.h properties.h xml2lalg.h > Log Message: > -- Extend Pathfinder (and the binaries pfopt and pfsql) to cope > with query plan bundles (generated by ferry). Multiple algebraic > plans bundled together form a query with a nested result type. > > We can now import the XML representation of a query plan bundle > and optimize all plans in the bundle. Furthermore we can generate > a DOT, an XML, and a SQL representation of the query plan bundle. > > NOTE: The XML import for 'normal' query plans as well as the code > and output generation still works as before. > > > U xml2lalg.h > Index: xml2lalg.h > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/compiler/include/xml2lalg.h,v > retrieving revision 1.6 > retrieving revision 1.7 > diff -u -d -r1.6 -r1.7 > --- xml2lalg.h 8 Jan 2009 16:54:12 -0000 1.6 > +++ xml2lalg.h 7 May 2009 20:00:16 -0000 1.7 > @@ -39,6 +39,7 @@ > #define XML2ALG_H > > #include "logical.h" > +#include "plan_bundle.h" > > #include "array.h" > > @@ -53,10 +54,10 @@ > XML2LALGContext* > PFxml2la_xml2lalgContext(void); > > -PFla_op_t* > +PFla_pb_t* > PFxml2la_importXMLFromFile(XML2LALGContext* ctx, const char *filename); > > -PFla_op_t* > +PFla_pb_t* > PFxml2la_importXMLFromMemory(XML2LALGContext* ctx, const char* xml, int size); > > #endif > > U logdebug.h > Index: logdebug.h > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/compiler/include/logdebug.h,v > retrieving revision 1.9 > retrieving revision 1.10 > diff -u -d -r1.9 -r1.10 > --- logdebug.h 8 Jan 2009 16:54:12 -0000 1.9 > +++ logdebug.h 7 May 2009 20:00:16 -0000 1.10 > @@ -4,7 +4,7 @@ > * @file > * > * Declarations for logdebug.c; dump Algebra > - * tree in AY&T dot format. > + * tree in AT&T dot or XML format. > * > * > * Copyright Notice: > @@ -43,10 +43,15 @@ > #include <stdio.h> > > #include "logical.h" > +#include "plan_bundle.h" > > void PFla_dot (FILE *, PFla_op_t *, char *prop_args); > void PFla_xml (FILE *, PFla_op_t *, char *prop_args); > > +/* Declaration for debug printing of plan bundles */ > +void PFla_dot_bundle (FILE *, PFla_pb_t *, char *prop_args); > +void PFla_xml_bundle (FILE *, PFla_pb_t *, char *prop_args); > + > #endif /* LOGDEBUG_H */ > > /* vim:set shiftwidth=4 expandtab: */ > > U properties.h > Index: properties.h > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/compiler/include/properties.h,v > retrieving revision 1.51 > retrieving revision 1.52 > diff -u -d -r1.51 -r1.52 > --- properties.h 31 Jan 2009 00:01:15 -0000 1.51 > +++ properties.h 7 May 2009 20:00:16 -0000 1.52 > @@ -374,7 +374,7 @@ > * Write domain-subdomain relationships of property container @a prop > * to in AT&T dot notation to character array @a f. > */ > -void PFprop_write_dom_rel_dot (PFarray_t *f, const PFprop_t *prop); > +void PFprop_write_dom_rel_dot (PFarray_t *f, const PFprop_t *prop, int id); > > /** > * Write domain-subdomain relationships of property container @a prop > > U Makefile.ag > Index: Makefile.ag > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/compiler/include/Makefile.ag,v > retrieving revision 1.52 > retrieving revision 1.53 > diff -u -d -r1.52 -r1.53 > --- Makefile.ag 8 Jan 2009 16:54:12 -0000 1.52 > +++ Makefile.ag 7 May 2009 20:00:16 -0000 1.53 > @@ -82,6 +82,7 @@ > physical.h \ > physical_mnemonic.h \ > planner.h \ > + plan_bundle.h \ > prettyp.h \ > properties.h \ > qname.h \ > > > ------------------------------------------------------------------------------ > The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your > production scanning environment may not be a perfect world - but thanks to > Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700 > Series Scanner you'll get full speed at 300 dpi even with all image > processing features enabled. http://p.sf.net/sfu/kodak-com > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

[Monetdb-developers] Workaround for JDK6 Bug 6795561
by Bram Van Dam 07 May '09

07 May '09

Howdy, When using the MonetDB JDBC driver on JDK6 (or JRE6) > 6.10 you will get IndexOutOfBoundsExceptions when using transactions (possibly in other cases as well). This happens because there is a bug in CharBuffer.subSequence(), which is used in StartOfHeaderParser.java The following patch remedies this problem, without any regressions as far as I can tell. Recompiling the JDBC driver is all that's required. You'll still have to compile with the 1.5 option because the JDBC interface changed quite a bit between 5 and 6, but if I have time left I might fix that as well. Hope this'll help someone. --- MonetDB-java/java/src/nl/cwi/monetdb/mcl/parser/StartOfHeaderParser.java Wed Mar 04 12:21:50 2009 +++ MonetDB-java/java/src/nl/cwi/monetdb/mcl/parser/StartOfHeaderParser.java Thu May 07 11:57:47 2009 @@ -141,8 +141,7 @@ while (soh.hasRemaining() && soh.get() != ' ') { cnt++; } - soh.reset(); - return(soh.subSequence(0, cnt).toString()); + return(soh.slice().subSequence(0, cnt).toString()); } }

3 2

Re: [Monetdb-developers] Compiling pathfinder (Stable) failed on GNU.32.32.d.1-Debian4.0
by Stefan Manegold 07 May '09

07 May '09

On Fri, May 01, 2009 at 05:56:16AM +0200, Testweb Compilation wrote: > from lex.pf.c:19: > /usr/include/features.h:150:1: error: this is the location of the previous definition > make[5]: *** [libparser_la-scanner.yy.lo] Error 1 > > http://www.ins.cwi.nl/~monet/permastore/2009-05-01/Stable/make-pathfinder-G… > > CVS committers: fabian(a)cwi.nl manegold(a)cwi.nl > > module maintainers: lsidir(a)cwi.nl zhang(a)cwi.nl henning(a)cwi.nl The problem here is that in pathfinder, pf_config.h is in most C files (possibly at least all C files in pathfinder/compiler/ not included directly, but via pathfinder/compilier/include/pathfinder.h . Hence, our "trick" in buildtools/conf/rules.mk, to push the inclusion of pf_config.h (also) for flex, bison, swig, etc. -generated C files to the veru front of the file, does not work properly, i.e., pf_config.h get includes twice, once "forced" directly at the beginning of the (generated) C file, and once via pathfinder.h . I see three solutions: 1) Remove '#include "pf_config.h"' from pathfinder/compilier/include/pathfinder.h and add '#include "pf_config.h"' as first include in all source files in pathfinder// (e.g., just before the existing '#include "pathfinder.h"'. 2) Make buildtools/conf/rules.mk, aware of pathfinder.h and handle is just like $(CONFIG_H) 3) (Try to) make bootstrap (i.e., auto*) a conf/config.h.in (or make configure create the <pkg>_config.h from conf/config.h.in) in such a way that it is resistent against multiple inclusion, e.g., using the usual #ifndef #define ... #endif mechanism. I have a strong preference for solution one, as that is what we do the the remainder of the code base. Any comments? Any volunteers? Stefan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 1

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/modules/mal remote.mx, , 1.95, 1.96
by Sjoerd Mullender 07 May '09

07 May '09

Martin Kersten wrote: > Update of /cvsroot/monetdb/MonetDB5/src/modules/mal > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv8081 > > Modified Files: > remote.mx > Log Message: > remote - simplified the RMT connection setup > in RMTquery first check for connnection errors, otherwise the handler > seems rubish. Question, is mapi_query() thread safe? I seriously doubt it. Use multiple connection handles if you want to use multiple threads. We could look into thread safety in mapilib sometime. -- Sjoerd Mullender

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/modules/mal pcre.mx, , 1.77, 1.78
by Stefan Manegold 06 May '09

06 May '09

Peter, I do not like this change very much. To ensure portability, we require that <PKG>_config.h (here: mal_config.h) is the very first include in each C file. Moving '#include "mal_config.h"' behind '#include "mal_config.h"' might cause unintended/unexpected problems now or in the future on other systems than Red Hat 4.3.2-7. Likewise, a hardwired and unconditional '#define _GNU_SOURCE 1' will not work on all systems. If Red Hat 4.3.2-7 indeed needs some special treatment, we should better try to fix our configure script(s) accordingly than have these hardcode fixes in the code itself. Stefan On Wed, May 06, 2009 at 01:10:59PM +0000, Peter Boncz wrote: > Update of /cvsroot/monetdb/MonetDB5/src/modules/mal > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv1809 > > Modified Files: > pcre.mx > Log Message: > make it compile on Red Hat 4.3.2-7 > > > > U pcre.mx > Index: pcre.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/modules/mal/pcre.mx,v > retrieving revision 1.77 > retrieving revision 1.78 > diff -u -d -r1.77 -r1.78 > --- pcre.mx 4 May 2009 13:53:25 -0000 1.77 > +++ pcre.mx 6 May 2009 13:10:53 -0000 1.78 > @@ -123,9 +123,10 @@ > @+ Implementation > @include prelude.mx > @c > -#include "mal_config.h" > +#define _GNU_SOURCE 1 > #include <string.h> > > +#include "mal_config.h" > #include "mal.h" > #include "mal_exception.h" > > @@ -142,6 +143,7 @@ > > #include <pcre.h> > > + > pcre_export str PCREquote(str *r, str *v); > pcre_export str PCREselect(int *res, str *pattern, int *bid); > pcre_export str PCREuselect(int *res, str *pattern, int *bid); > > > ------------------------------------------------------------------------------ > The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your > production scanning environment may not be a perfect world - but thanks to > Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700 > Series Scanner you'll get full speed at 300 dpi even with all image > processing features enabled. http://p.sf.net/sfu/kodak-com > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 1

[Monetdb-developers] removing a BAT from C for real
by Fabian Groffen 05 May '09

05 May '09

Can someone tell me how to remove a BAT in MonetDB from C code for good and for REAL? I'll probably miss something trivial, as I tried about 20 combinations of how to get rid of a previously persistent BAT, but everytime I restart my server it comes back again like a BIOS virus. Place in the source: monetdb5/src/mal/mal_authorize.mx, I want to remove an old M5system_auth_* BAT after I created a new one with converted data in the initTables function. My latest incarnation looks like this (copied and pasted from destroyImmediate): BBPlogical(b->batCacheid, buf); BBPreleaseref(b->batCacheid); BBPfix(bid); if (BBPindex(buf) == bid) { BAT *t = (BAT*)BBPgetdesc(ABS(bid)); BATmode(t, TRANSIENT); } BBPunfix(bid); said BAT looks gone in the session, but returns next time the server starts. Can I (sub)commit the removed BAT? Any hint on this strong persistence would be nice.

1 1

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/modules/mal remote.mx, , 1.96, 1.97
by Martin Kersten 05 May '09

05 May '09

Fabian Groffen wrote: > On 05-05-2009 12:31:13 +0000, Martin Kersten wrote: >> - if (mapi_error(conn) != MOK) >> + if (mapi_error(conn) != MOK){ >> + mapi_explain(conn,stderr); > > better ifdef such statements IMO... # MonetDB/SQL module v2.29.0 loaded [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] MAPI = monetdb@127.0.0.1:50000 REMOTE GET error: io.print(rmt_octopus_exec__9_bat__oid__int_); IOException:remote.get:an error occurred on connection: (null) !MALException:remote.get:IOException:remote.get:an error occurred on connection: (null) Dit is wat er dus gebeurd in RMTget -> RMTquery is herhaalbaar, maar in iedergeval een gevolg van parallel processing, onduidelijk waarom de connectie er al uitgaat. > > > ------------------------------------------------------------------------------ > The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your > production scanning environment may not be a perfect world - but thanks to > Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700 > Series Scanner you'll get full speed at 300 dpi even with all image > processing features enabled. http://p.sf.net/sfu/kodak-com > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/modules/mal remote.mx, , 1.96, 1.97
by Martin Kersten 05 May '09

05 May '09

Fabian Groffen wrote: > On 05-05-2009 12:31:13 +0000, Martin Kersten wrote: >> - if (mapi_error(conn) != MOK) >> + if (mapi_error(conn) != MOK){ >> + mapi_explain(conn,stderr); > > better ifdef such statements IMO... ja..... probeer die race conditie te pinnen.... :(( > > > ------------------------------------------------------------------------------ > The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your > production scanning environment may not be a perfect world - but thanks to > Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700 > Series Scanner you'll get full speed at 300 dpi even with all image > processing features enabled. http://p.sf.net/sfu/kodak-com > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins

1 0