- developers-list - monetdb.org

bat bind Access kind
by Tapomay Dey 10 Dec '12

10 Dec '12

pattern bind(mvc:int, schema:str, table:str, column:str, access:int ):bat[:oid,:any_1] address mvc_bind_wrap comment "Bind the 'schema.table.column' BAT with access kind: 0 - base table 1 - inserts 2 - updates"; What do you mean by access kind here. Appreciate any help. Regards, Tapomay. _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

2 2

Re: MonetDB: Feb2013 - Introduce a sink point for parallel processing
by Fabian Groffen 10 Dec '12

10 Dec '12

On 10-12-2012 11:49:29 +0100, Martin Kersten wrote: > Changeset: d944146a71ef for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=d944146a71ef > Modified Files: > monetdb5/modules/mal/language.c > monetdb5/modules/mal/language.h > monetdb5/modules/mal/language.mal > Branch: Feb2013 > Log Message: > > Introduce a sink point for parallel processing sink noun 1 a fixed basin with a water supply and a drain. 2 a pool or marsh in which a river's water disappears by evaporation or percolation. • technical a body or process that acts to absorb or remove energy or a particular component from a system: a heat sink | the oceans can act as a sink for CO 2. The opposite of source. 3 short for sinkhole. 4 a place of vice or corruption: moralizers complain that the Net is simply a festering sink of iniquity. sync |siNGk|(also synch ) informal noun synchronization: images flash onto your screen in sync with the music. -- Fabian Groffen fabian(a)monetdb.org column-store pioneer http://www.monetdb.org/Home _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

1 0

Re: MonetDB: default - use new sub sample function (cleans up some m...
by Sjoerd Mullender 06 Dec '12

06 Dec '12

Why the mix of ptr/BUN/wrd in SAMPLEsubuniform? The function is only called from MAL for a command whose last parameter is declared as s:wrd, so SAMPLEsubuniform can be declared as having a last argument as wrd *s and the cast inside then becomes correct too: (BUN) *s. Remember, wrd and BUN are not *necessarily* the same size. On 2012-12-06 10:25, Niels Nes wrote: > Changeset: 7c832c8b1a70 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=7c832c8b1a70 > Modified Files: > monetdb5/modules/kernel/algebra.mx > monetdb5/modules/mal/sample.c > monetdb5/modules/mal/sample.h > monetdb5/modules/mal/sample.mal > sql/backends/monet5/rel_bin.c > sql/backends/monet5/sql_gencode.c > Branch: default > Log Message: > > use new sub sample function (cleans up some more reverses and marks) > > > diffs (112 lines): > > diff --git a/monetdb5/modules/kernel/algebra.mx b/monetdb5/modules/kernel/algebra.mx > --- a/monetdb5/modules/kernel/algebra.mx > +++ b/monetdb5/modules/kernel/algebra.mx > @@ -282,6 +282,11 @@ comment "Selects all elements that have > command sample ( b:bat[:oid,:any_2], num:int ) :bat[:oid,:any_2] > address ALGsample > comment "Produce a random selection of size 'num' from the input BAT."; > + > +command subsample(b:bat[:oid,:any_1], num:int ) :bat[:oid,:oid] > +address ALGsubsample > +comment "Return the oids of a random selection of size 'num' from the input BAT."; > + > # @+ BAT copying > command copy( b:bat[:any_1,:any_2]) :bat[:any_1,:any_2] > address ALGcopy > @@ -1076,6 +1081,7 @@ algebra_export str ALG@1(int *result, in > @:ALGbinaryExport(tdifference)@ > @:ALGbinaryExport(tdiff)@ > @:ALGbinaryintExport(sample)@ > +@:ALGbinaryintExport(subsample)@ > > algebra_export str ALGtunique(int *result, int *bid); > algebra_export str ALGtsort(int *result, int *bid); > @@ -2765,6 +2771,11 @@ ALGsample(bat *result, bat *bid, int *pa > return ALGbinaryint(result, bid, param, BATsample, "algebra.sample"); > } > > +str > +ALGsubsample(bat *result, bat *bid, int *param) > +{ > + return ALGbinaryint(result, bid, param, BATsample_, "algebra.subsample"); > +} > > /* add items missing in the kernel */ > str > diff --git a/monetdb5/modules/mal/sample.c b/monetdb5/modules/mal/sample.c > --- a/monetdb5/modules/mal/sample.c > +++ b/monetdb5/modules/mal/sample.c > @@ -96,6 +96,23 @@ SAMPLEuniform(bat *r, bat *b, ptr s) { > } > > sample_export str > +SAMPLEsubuniform(bat *r, bat *b, ptr s) { > + BAT *br, *bb; > + > + if ((bb = BATdescriptor(*b)) == NULL) { > + throw(MAL, "sample.subuniform", INTERNAL_BAT_ACCESS); > + } > + br = BATsample_(bb,*(BUN *)s); > + if (br == NULL) > + throw(MAL, "sample.subuniform", OPERATION_FAILED); > + > + BBPunfix(bb->batCacheid); > + BBPkeepref(*r = br->batCacheid); > + return MAL_SUCCEED; > + > +} > + > +sample_export str > SAMPLEuniform_dbl(bat *r, bat *b, ptr p) { > BAT *bb; > double pr = *(double *)p; > diff --git a/monetdb5/modules/mal/sample.h b/monetdb5/modules/mal/sample.h > --- a/monetdb5/modules/mal/sample.h > +++ b/monetdb5/modules/mal/sample.h > @@ -42,6 +42,9 @@ sample_export str > SAMPLEuniform(bat *r, bat *b, ptr s); > > sample_export str > +SAMPLEsubuniform(bat *r, bat *b, ptr s); > + > +sample_export str > SAMPLEuniform_dbl(bat *r, bat *b, ptr p); > > #endif > diff --git a/monetdb5/modules/mal/sample.mal b/monetdb5/modules/mal/sample.mal > --- a/monetdb5/modules/mal/sample.mal > +++ b/monetdb5/modules/mal/sample.mal > @@ -27,6 +27,10 @@ command uniform(b:bat[:oid,:any],s:wrd): > address SAMPLEuniform > comment "Returns a uniform sample of size s" > > +command subuniform(b:bat[:oid,:any],s:wrd):bat[:oid,:oid] > +address SAMPLEsubuniform > +comment "Returns the oids of a uniform sample of size s" > + > command uniform(b:bat[:oid,:any],p:dbl):bat[:oid,:any] > address SAMPLEuniform_dbl > comment "Returns a uniform sample of size = (p x count(b)), where 0 <= p <= 1.0" > diff --git a/sql/backends/monet5/rel_bin.c b/sql/backends/monet5/rel_bin.c > --- a/sql/backends/monet5/rel_bin.c > +++ b/sql/backends/monet5/rel_bin.c > @@ -2677,7 +2677,6 @@ rel2bin_sample( mvc *sql, sql_rel *rel, > sc = column(sql->sa, sc); > sample = stmt_sample(sql->sa, stmt_alias(sql->sa, sc, tname, cname),s); > > - sample = stmt_reverse(sql->sa, stmt_mark_tail(sql->sa, sample, 0)); > for ( ; n; n = n->next) { > stmt *sc = n->data; > char *cname = column_name(sql->sa, sc); > diff --git a/sql/backends/monet5/sql_gencode.c b/sql/backends/monet5/sql_gencode.c > --- a/sql/backends/monet5/sql_gencode.c > +++ b/sql/backends/monet5/sql_gencode.c > @@ -983,7 +983,7 @@ _dumpstmt(backend *sql, MalBlkPtr mb, st > case st_sample:{ > int l = _dumpstmt(sql, mb, s->op1); > int r = _dumpstmt(sql, mb, s->op2); > - q = newStmt(mb, "sample", "uniform"); > + q = newStmt(mb, "sample", "subuniform"); > q = pushArgument(mb, q, l); > q = pushArgument(mb, q, r); > s->nr = getDestVar(q); > _______________________________________________ > checkin-list mailing list > checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list > -- Sjoerd Mullender _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

1 0

Re: MonetDB: Oct2012 - autogen: fix for Python 3
by Stefan Manegold 04 Dec '12

04 Dec '12

This (or previous?) checkin(s) seem to break compilation with python 2.7.3: Making all in python make[4]: Entering directory `/net/rome.ins.cwi.nl/export/scratch1/manegold/Monet/HG/Oct2012/builds/--disable-debug_--enable-assert_--enable-optimize/clients/python' make all-am make[5]: Entering directory `/net/rome.ins.cwi.nl/export/scratch1/manegold/Monet/HG/Oct2012/builds/--disable-debug_--enable-assert_--enable-optimize/clients/python' [ '/ufs/manegold/_/Monet/HG/Oct2012/source/MonetDB/clients/python' -ef . ] || cp -p '/ufs/manegold/_/Monet/HG/Oct2012/source/MonetDB/clients/python/setup.py' 'setup.py' [ '/ufs/manegold/_/Monet/HG/Oct2012/source/MonetDB/clients/python' -ef . ] || mkdir -p '['monetdb']' [ '/ufs/manegold/_/Monet/HG/Oct2012/source/MonetDB/clients/python' -ef . ] || cp -p '/ufs/manegold/_/Monet/HG/Oct2012/source/MonetDB/clients/python/['monetdb']'/*.py '['monetdb']' cp: cannot stat `/ufs/manegold/_/Monet/HG/Oct2012/source/MonetDB/clients/python/[monetdb]/*.py': No such file or directory make[5]: *** [all-local-python_setup] Error 1 make[5]: Leaving directory `/net/rome.ins.cwi.nl/export/scratch1/manegold/Monet/HG/Oct2012/builds/--disable-debug_--enable-assert_--enable-optimize/clients/python' make[4]: *** [all] Error 2 make[4]: Leaving directory `/net/rome.ins.cwi.nl/export/scratch1/manegold/Monet/HG/Oct2012/builds/--disable-debug_--enable-assert_--enable-optimize/clients/python' make[3]: *** [all-recursive] Error 1 make[3]: Leaving directory `/net/rome.ins.cwi.nl/export/scratch1/manegold/Monet/HG/Oct2012/builds/--disable-debug_--enable-assert_--enable-optimize/clients' make[2]: *** [all] Error 2 make[2]: Leaving directory `/net/rome.ins.cwi.nl/export/scratch1/manegold/Monet/HG/Oct2012/builds/--disable-debug_--enable-assert_--enable-optimize/clients' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/net/rome.ins.cwi.nl/export/scratch1/manegold/Monet/HG/Oct2012/builds/--disable-debug_--enable-assert_--enable-optimize' make: *** [all] Error 2 Stefan On Tue, Dec 04, 2012 at 09:08:46PM +0100, Fabian Groffen wrote: > Changeset: d46e57d76fec for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=d46e57d76fec > Modified Files: > buildtools/autogen/autogen.py > buildtools/autogen/autogen/am.py > buildtools/autogen/autogen/codegen.py > buildtools/autogen/autogen/msc.py > buildtools/autogen/autogen/var.py > Branch: Oct2012 > Log Message: > > autogen: fix for Python 3 > [...] -- | Stefan.Manegold @ CWI.nl | DB Architectures (INS1) | | http://CWI.nl/~manegold/ | Science Park 123 (L321) | | Tel.: +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) | _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

3 4

Dear MonetDB development team. I have some question for MonetDB
by 김경호 03 Dec '12

03 Dec '12

Dear MonetDB development team. I am a graduate student attending Korea's Hongik University. I am working on a project that improving query processing performance. I want to add some algorithms in MonetDB source code. Can I ask some questions by this e-mail address? Yours sincerely. fantastictoy님의 블로그 안녕하세요. _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

3 2

Re: MonetDB: default - allow fixed number of partitions using --set ...
by Fabian Groffen 02 Dec '12

02 Dec '12

On 02-12-2012 11:51:30 +0100, Niels Nes wrote: > Changeset: 6e166ce6744e for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=6e166ce6744e > Modified Files: > gdk/gdk_utils.c > gdk/gdk_utils.h > monetdb5/optimizer/opt_mitosis.c > Branch: default > Log Message: > > allow fixed number of partitions using --set mito_parts=? isn't this the same as --forcemito + gdk_nr_threads=? -- Fabian Groffen fabian(a)monetdb.org column-store pioneer http://www.monetdb.org/Home _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

2 1

Octopus and
by Masood Mortazavi 02 Dec '12

02 Dec '12

Hi, Where would I find any documentation or instructions to try MonetDB's Octopus functionality, assuming it is still active at (or near) the tip? Regards, Masood P.S. I would also be interested in exploring Monet's any other distributed processing functionality, say Monet/Datacyclotron or Monet/MapReduce. _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

1 1

New Feb2013 stable branch created
by Sjoerd Mullender 29 Nov '12

29 Nov '12

I have created a new stable branch for the next feature release. This branch was created in order to stabilize the code before the feature release. From now on *only* bug fixes may go to this new stable branch. I reserve the right to undo any changes that do not comply with this rule. I also reserve the right to admit changes that are not strictly bug fixes. The name for the branch is Feb2013. So in each of your stable checkout directories you can do hg pull hg update -rFeb2013 to update to this new branch. -- Sjoerd Mullender _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

1 0

Announcement: New Oct2012-SP1 Bugfix release of MonetDB suite
by Sjoerd Mullender 21 Nov '12

21 Nov '12

The MonetDB team at CWI/MonetDB BV is pleased to announce the Oct2012-SP1 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <http://www.monetdb.org/>. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Oct 2012-SP1 bugfix release Bug Fixes * 3155: Provide an option for setting discovery to false * 3161: invalid tuple received from server, got 138 columns, expected 476, ignoring * 3182: When using aggregate SQL functions, precision & decimal metadata for numeric/decimal type returned is 0,0 _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

1 0

[Bug 3178] one scan is enough to implement ALGstdev_@1 in monetdb5/modules/kernel/algebra.mx
by Yinhe Cao 20 Nov '12

20 Nov '12

Hi Fabian, Thank you very much for your email. I had the same concern about the accumulation error of one-scan approach as you do, even though mathematically they should be exactly the same. I remember about 10 years ago, I did some experimentations on both of the approaches and I did not found any noticeable differences between the results, but the sample size I tried then is not huge. For the sample size of the order 1 trillion, I have the same concern even with the two-scan approach. For example, After 500 billion entries have been added to the sum of square, the rest may not be able to be added up anymore, since it may well with the range of accumulation errors! For a huge BAT, if the precision is top priority, we may need to use an hierarchical approach: divide the huge sample into some manageable block, compute it for each block first, and then aggregate the results. Thanks, Yinhe _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

1 0