December 2012 - developers-list

group_concat
by Tapomay Dey 13 Dec '12

13 Dec '12

Almost there. As suggested by Fabian in the following thread I gave up the MX file approach and implemented group_concat as UDF in C. http://sourceforge.net/mailarchive/forum.php?thread_name=mailman.243.135532… When I try to conclude my function UDFBATgroup_concat_(BAT **ret, BAT *b, BAT *g, BAT *h, FILE* fp) //ignore fp as follows: BATaccessBegin(h, USE_HEAD | USE_TAIL, MMAP_SEQUENTIAL); index = 0; BATloop(h, p, q) { ptr hhead = BUNhead(hi, p); fprintf(fp, "%d: hhead=%d", index, hhead); str dst = tailValArray[index]; fprintf(fp, "%d: dstStr=%s# ", index, dst); BUNins(bn, hhead, dst, FALSE); fprintf(fp, "%d: Insert complete\n", index); index++; } I get the following error on the client terminal [tapomay@tapomay-laptop monetdata]$ echo "select house, group_concat(name) from gs group by house;" | mclient -d tapomay GDKerror:!ERROR: BUNins: access denied to tmp_10162, aborting. BUNins: access denied to tmp_10162, aborting. BUNins: access denied to tmp_10162, aborting. The groupings are ready in the tailVarArray. NEED HELP. Thanks and Regards, Tapomay. _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

2 1

Still searching to find the Octopus :-)
by Masood Mortazavi 12 Dec '12

12 Dec '12

I'm still searching for user documentation that would explain MonetDB/octopus usage/feature. Does any exist? Thanks, Masood P.S. My earlier e-mails regarding the octopus documentation was somewhat rushed, it seems. The documentation in the man pages are actually for remtoe access and funnel-multiplex features, which seem entirely different. _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

2 1

MX file expansion
by Tapomay Dey 12 Dec '12

12 Dec '12

How do I generate the sources from a new mx file. Thanks and Regards, Tapomay _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

3 5

Re: MonetDB: Oct2012 - configure fixes for if there is no python2 bi...
by Fabian Groffen 11 Dec '12

11 Dec '12

On 11-12-2012 15:59:04 +0100, Sjoerd Mullender wrote: > Changeset: fb7305c917ec for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=fb7305c917ec > Modified Files: > configure.ag > Branch: Oct2012 > Log Message: > > configure fixes for if there is no python2 binary, but python is version 2. > Also fixed a syntax error. > +AC_PATH_PROG(PYTHON,python,no,$PATH) > +PYTHON_MAJ= > +if test "x$PYTHON" != xno; then > + AC_MSG_CHECKING([major version of $PYTHON]) > + case "`"$PYTHON" -V 2>&1`" in > + "Python 2."[[67]]*) # older Pythons don't get Python3 syntax > + PYTHON_MAJ=2 > + case "$have_python2" in > + yes|auto) > + have_python2=yes > + PYTHON2="$PYTHON" Because PYTHON2=/path/to/python here, we no longer check for python2 which is preferably over "/path/to/python", since it contains a version. The active python can change over updates or preferences of the user. Perhaps (something like) the following would be enough? diff --git a/configure.ag b/configure.ag --- a/configure.ag +++ b/configure.ag @@ -1164,7 +1164,7 @@ fi if test "x$have_python2" != xno; then if test x$cross_compiling != xyes; then - AC_PATH_PROG(PYTHON2,$PYTHON2,no,$PATH) + AC_PATH_PROG(PYTHON2,python2,no,$PATH) if test "x$PYTHON2" = xno; then if ( test "x$have_python2" = xyes || test "x$have_python2" = xauto ) && test "x$PYTHON_MAJ" = x2; then PYTHON2="$PYTHON" @@ -1220,7 +1220,7 @@ AC_SUBST(PYTHON2_LIBDIR) if test "x$have_python3" != xno; then if test x$cross_compiling != xyes; then - AC_PATH_PROG(PYTHON3,$PYTHON3,no,$PATH) + AC_PATH_PROG(PYTHON3,python3,no,$PATH) if test "x$PYTHON3" = xno; then if ( test "x$have_python3" = xyes || test "x$have_python3" = xauto ) && test "x$PYTHON_MAJ" = x3; then PYTHON3="$PYTHON" -- Fabian Groffen fabian(a)monetdb.org column-store pioneer http://www.monetdb.org/Home _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

2 3

Re: MonetDB: Feb2013 - dotmonetdb: don't support passwd
by Lefteris 11 Dec '12

11 Dec '12

I was using that:P On Sat, Dec 8, 2012 at 4:20 PM, Fabian Groffen <commits(a)monetdb.org> wrote: > Changeset: c1f634d9db8c for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=c1f634d9db8c > Modified Files: > clients/mapiclient/dotmonetdb.c > Branch: Feb2013 > Log Message: > > dotmonetdb: don't support passwd > > The purposely only option for it ever documented is "password", so no > need to check for more than that. > > > diffs (12 lines): > > diff --git a/clients/mapiclient/dotmonetdb.c > b/clients/mapiclient/dotmonetdb.c > --- a/clients/mapiclient/dotmonetdb.c > +++ b/clients/mapiclient/dotmonetdb.c > @@ -95,7 +95,7 @@ parse_dotmonetdb(char **user, char **pas > if (user) > *user = strdup(q); > q = NULL; > - } else if (strcmp(buf, "password") == 0 || > strcmp(buf, "passwd") == 0) { > + } else if (strcmp(buf, "password") == 0) == 0) { > if (passwd) > *passwd = strdup(q); > q = NULL; > _______________________________________________ > checkin-list mailing list > checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list > _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

1 0

bat bind Access kind
by Tapomay Dey 10 Dec '12

10 Dec '12

pattern bind(mvc:int, schema:str, table:str, column:str, access:int ):bat[:oid,:any_1] address mvc_bind_wrap comment "Bind the 'schema.table.column' BAT with access kind: 0 - base table 1 - inserts 2 - updates"; What do you mean by access kind here. Appreciate any help. Regards, Tapomay. _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

2 2

Re: MonetDB: Feb2013 - Introduce a sink point for parallel processing
by Fabian Groffen 10 Dec '12

10 Dec '12

On 10-12-2012 11:49:29 +0100, Martin Kersten wrote: > Changeset: d944146a71ef for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=d944146a71ef > Modified Files: > monetdb5/modules/mal/language.c > monetdb5/modules/mal/language.h > monetdb5/modules/mal/language.mal > Branch: Feb2013 > Log Message: > > Introduce a sink point for parallel processing sink noun 1 a fixed basin with a water supply and a drain. 2 a pool or marsh in which a river's water disappears by evaporation or percolation. • technical a body or process that acts to absorb or remove energy or a particular component from a system: a heat sink | the oceans can act as a sink for CO 2. The opposite of source. 3 short for sinkhole. 4 a place of vice or corruption: moralizers complain that the Net is simply a festering sink of iniquity. sync |siNGk|(also synch ) informal noun synchronization: images flash onto your screen in sync with the music. -- Fabian Groffen fabian(a)monetdb.org column-store pioneer http://www.monetdb.org/Home _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

1 0

Re: MonetDB: default - use new sub sample function (cleans up some m...
by Sjoerd Mullender 06 Dec '12

06 Dec '12

Why the mix of ptr/BUN/wrd in SAMPLEsubuniform? The function is only called from MAL for a command whose last parameter is declared as s:wrd, so SAMPLEsubuniform can be declared as having a last argument as wrd *s and the cast inside then becomes correct too: (BUN) *s. Remember, wrd and BUN are not *necessarily* the same size. On 2012-12-06 10:25, Niels Nes wrote: > Changeset: 7c832c8b1a70 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=7c832c8b1a70 > Modified Files: > monetdb5/modules/kernel/algebra.mx > monetdb5/modules/mal/sample.c > monetdb5/modules/mal/sample.h > monetdb5/modules/mal/sample.mal > sql/backends/monet5/rel_bin.c > sql/backends/monet5/sql_gencode.c > Branch: default > Log Message: > > use new sub sample function (cleans up some more reverses and marks) > > > diffs (112 lines): > > diff --git a/monetdb5/modules/kernel/algebra.mx b/monetdb5/modules/kernel/algebra.mx > --- a/monetdb5/modules/kernel/algebra.mx > +++ b/monetdb5/modules/kernel/algebra.mx > @@ -282,6 +282,11 @@ comment "Selects all elements that have > command sample ( b:bat[:oid,:any_2], num:int ) :bat[:oid,:any_2] > address ALGsample > comment "Produce a random selection of size 'num' from the input BAT."; > + > +command subsample(b:bat[:oid,:any_1], num:int ) :bat[:oid,:oid] > +address ALGsubsample > +comment "Return the oids of a random selection of size 'num' from the input BAT."; > + > # @+ BAT copying > command copy( b:bat[:any_1,:any_2]) :bat[:any_1,:any_2] > address ALGcopy > @@ -1076,6 +1081,7 @@ algebra_export str ALG@1(int *result, in > @:ALGbinaryExport(tdifference)@ > @:ALGbinaryExport(tdiff)@ > @:ALGbinaryintExport(sample)@ > +@:ALGbinaryintExport(subsample)@ > > algebra_export str ALGtunique(int *result, int *bid); > algebra_export str ALGtsort(int *result, int *bid); > @@ -2765,6 +2771,11 @@ ALGsample(bat *result, bat *bid, int *pa > return ALGbinaryint(result, bid, param, BATsample, "algebra.sample"); > } > > +str > +ALGsubsample(bat *result, bat *bid, int *param) > +{ > + return ALGbinaryint(result, bid, param, BATsample_, "algebra.subsample"); > +} > > /* add items missing in the kernel */ > str > diff --git a/monetdb5/modules/mal/sample.c b/monetdb5/modules/mal/sample.c > --- a/monetdb5/modules/mal/sample.c > +++ b/monetdb5/modules/mal/sample.c > @@ -96,6 +96,23 @@ SAMPLEuniform(bat *r, bat *b, ptr s) { > } > > sample_export str > +SAMPLEsubuniform(bat *r, bat *b, ptr s) { > + BAT *br, *bb; > + > + if ((bb = BATdescriptor(*b)) == NULL) { > + throw(MAL, "sample.subuniform", INTERNAL_BAT_ACCESS); > + } > + br = BATsample_(bb,*(BUN *)s); > + if (br == NULL) > + throw(MAL, "sample.subuniform", OPERATION_FAILED); > + > + BBPunfix(bb->batCacheid); > + BBPkeepref(*r = br->batCacheid); > + return MAL_SUCCEED; > + > +} > + > +sample_export str > SAMPLEuniform_dbl(bat *r, bat *b, ptr p) { > BAT *bb; > double pr = *(double *)p; > diff --git a/monetdb5/modules/mal/sample.h b/monetdb5/modules/mal/sample.h > --- a/monetdb5/modules/mal/sample.h > +++ b/monetdb5/modules/mal/sample.h > @@ -42,6 +42,9 @@ sample_export str > SAMPLEuniform(bat *r, bat *b, ptr s); > > sample_export str > +SAMPLEsubuniform(bat *r, bat *b, ptr s); > + > +sample_export str > SAMPLEuniform_dbl(bat *r, bat *b, ptr p); > > #endif > diff --git a/monetdb5/modules/mal/sample.mal b/monetdb5/modules/mal/sample.mal > --- a/monetdb5/modules/mal/sample.mal > +++ b/monetdb5/modules/mal/sample.mal > @@ -27,6 +27,10 @@ command uniform(b:bat[:oid,:any],s:wrd): > address SAMPLEuniform > comment "Returns a uniform sample of size s" > > +command subuniform(b:bat[:oid,:any],s:wrd):bat[:oid,:oid] > +address SAMPLEsubuniform > +comment "Returns the oids of a uniform sample of size s" > + > command uniform(b:bat[:oid,:any],p:dbl):bat[:oid,:any] > address SAMPLEuniform_dbl > comment "Returns a uniform sample of size = (p x count(b)), where 0 <= p <= 1.0" > diff --git a/sql/backends/monet5/rel_bin.c b/sql/backends/monet5/rel_bin.c > --- a/sql/backends/monet5/rel_bin.c > +++ b/sql/backends/monet5/rel_bin.c > @@ -2677,7 +2677,6 @@ rel2bin_sample( mvc *sql, sql_rel *rel, > sc = column(sql->sa, sc); > sample = stmt_sample(sql->sa, stmt_alias(sql->sa, sc, tname, cname),s); > > - sample = stmt_reverse(sql->sa, stmt_mark_tail(sql->sa, sample, 0)); > for ( ; n; n = n->next) { > stmt *sc = n->data; > char *cname = column_name(sql->sa, sc); > diff --git a/sql/backends/monet5/sql_gencode.c b/sql/backends/monet5/sql_gencode.c > --- a/sql/backends/monet5/sql_gencode.c > +++ b/sql/backends/monet5/sql_gencode.c > @@ -983,7 +983,7 @@ _dumpstmt(backend *sql, MalBlkPtr mb, st > case st_sample:{ > int l = _dumpstmt(sql, mb, s->op1); > int r = _dumpstmt(sql, mb, s->op2); > - q = newStmt(mb, "sample", "uniform"); > + q = newStmt(mb, "sample", "subuniform"); > q = pushArgument(mb, q, l); > q = pushArgument(mb, q, r); > s->nr = getDestVar(q); > _______________________________________________ > checkin-list mailing list > checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list > -- Sjoerd Mullender _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

1 0

Re: MonetDB: Oct2012 - autogen: fix for Python 3
by Stefan Manegold 04 Dec '12

04 Dec '12

This (or previous?) checkin(s) seem to break compilation with python 2.7.3: Making all in python make[4]: Entering directory `/net/rome.ins.cwi.nl/export/scratch1/manegold/Monet/HG/Oct2012/builds/--disable-debug_--enable-assert_--enable-optimize/clients/python' make all-am make[5]: Entering directory `/net/rome.ins.cwi.nl/export/scratch1/manegold/Monet/HG/Oct2012/builds/--disable-debug_--enable-assert_--enable-optimize/clients/python' [ '/ufs/manegold/_/Monet/HG/Oct2012/source/MonetDB/clients/python' -ef . ] || cp -p '/ufs/manegold/_/Monet/HG/Oct2012/source/MonetDB/clients/python/setup.py' 'setup.py' [ '/ufs/manegold/_/Monet/HG/Oct2012/source/MonetDB/clients/python' -ef . ] || mkdir -p '['monetdb']' [ '/ufs/manegold/_/Monet/HG/Oct2012/source/MonetDB/clients/python' -ef . ] || cp -p '/ufs/manegold/_/Monet/HG/Oct2012/source/MonetDB/clients/python/['monetdb']'/*.py '['monetdb']' cp: cannot stat `/ufs/manegold/_/Monet/HG/Oct2012/source/MonetDB/clients/python/[monetdb]/*.py': No such file or directory make[5]: *** [all-local-python_setup] Error 1 make[5]: Leaving directory `/net/rome.ins.cwi.nl/export/scratch1/manegold/Monet/HG/Oct2012/builds/--disable-debug_--enable-assert_--enable-optimize/clients/python' make[4]: *** [all] Error 2 make[4]: Leaving directory `/net/rome.ins.cwi.nl/export/scratch1/manegold/Monet/HG/Oct2012/builds/--disable-debug_--enable-assert_--enable-optimize/clients/python' make[3]: *** [all-recursive] Error 1 make[3]: Leaving directory `/net/rome.ins.cwi.nl/export/scratch1/manegold/Monet/HG/Oct2012/builds/--disable-debug_--enable-assert_--enable-optimize/clients' make[2]: *** [all] Error 2 make[2]: Leaving directory `/net/rome.ins.cwi.nl/export/scratch1/manegold/Monet/HG/Oct2012/builds/--disable-debug_--enable-assert_--enable-optimize/clients' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/net/rome.ins.cwi.nl/export/scratch1/manegold/Monet/HG/Oct2012/builds/--disable-debug_--enable-assert_--enable-optimize' make: *** [all] Error 2 Stefan On Tue, Dec 04, 2012 at 09:08:46PM +0100, Fabian Groffen wrote: > Changeset: d46e57d76fec for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=d46e57d76fec > Modified Files: > buildtools/autogen/autogen.py > buildtools/autogen/autogen/am.py > buildtools/autogen/autogen/codegen.py > buildtools/autogen/autogen/msc.py > buildtools/autogen/autogen/var.py > Branch: Oct2012 > Log Message: > > autogen: fix for Python 3 > [...] -- | Stefan.Manegold @ CWI.nl | DB Architectures (INS1) | | http://CWI.nl/~manegold/ | Science Park 123 (L321) | | Tel.: +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) | _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

3 4

Dear MonetDB development team. I have some question for MonetDB
by 김경호 03 Dec '12

03 Dec '12

Dear MonetDB development team. I am a graduate student attending Korea's Hongik University. I am working on a project that improving query processing performance. I want to add some algorithms in MonetDB source code. Can I ask some questions by this e-mail address? Yours sincerely. fantastictoy님의 블로그 안녕하세요. _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

3 2