March 2016 - developers-list

Announcement: New Jul2015-SP3 Bugfix release of MonetDB suite
by Sjoerd Mullender 25 Mar '16

25 Mar '16

The MonetDB team at CWI/MonetDB BV is pleased to announce the Jul2015-SP3 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <http://www.monetdb.org/>. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Jul 2015-SP3 bugfix release MonetDB5 Server * Fixed potential crash in MAL debugger when accessing BATs by index. Functionality dropped as it is also a security leak. Bug Fixes * 2972: SQL URL functionality contains errors * 3549: bulk string operations very slow * 3881: Server crashes on bulk load * 3890: Window function + group by in subselect, rel2bin_project: Assertion `0' failed * 3891: MonetDB crashes when executing SQL with window function * 3900: null handling in some sql statements is incorrect * 3906: Multi-column 1-N table-function with mitosis produces different column counts * 3908: LEFT JOIN with OR conditions triggers assertion * 3909: Incorrect column name in OR condition of LEFT JOIN crashes mserver * 3910: COPY INTO table (column1, column2) got wrong result * 3912: When table/column names conflicts, data ends in multiple tables! * 3917: Date difference returns month_interval instead of day_interval * 3918: MonetDB.R version 1.0.1 incorrectly constructs the batfile script * 3919: Table conflict when the table name and fields are identical * 3921: Creating a table from a complex query crashes mserver or triggers assertion * 3922: AVG( column ) returns NaN rather than Inf when column contains Inf * 3928: When killing a virtual machine, sql_logs/sql/log is empty * 3930: Wrong typecast on character columns in prepared statements when using Umlaute * 3932: CASE expressions with constants are not evaluated correctly * 3933: replace "exit" by "throw new Exception" * 3937: bad BAT properties with binary copy into and NULL values * 3938: Wrong error message on violating foreign key constraint * 3940: Date calculation and comparison produce wrong result * 3941: Wrong coercion priority * 3948: SQL: select * from sys.sys.table_name; is accepted but should return an error * 3951: extern table_funcs not visible from Windows DLL for extensions like vaults (crashes) * 3952: Stream table gives segfault * 3953: MIN/MAX of a UUID column produces wrong results * 3954: Consolidate table assertion error * 3955: (incorrect) MAL loop instead of manifold triggered by simple change in target list

1 0

histogram
by Roberto Cornacchia 21 Mar '16

21 Mar '16

In a C UDF, looping on a [:oid,:str:] bat, I'm tokenizing each string tail into a str array: BATloop(..) { str *token_array = /* create a str array by tokenizing the str tail of this BUN */ /* append a histogram of token_array to result */ } The token array is expected to be in the range of 10 - 10K short strings (1 to 10 bytes). Question: To get the histogram, I'd turn the token_array into a BAT b, then use BATgroup: BATgroup(&gn, NULL, &hn, b, NULL, NULL, NULL); Do you see a more efficient way? Is there a group/histogram primitive implemented that works directly on arrays rather than BATs? Thanks, Roberto

1 0

Error after compiling monetdb
by Shmagi Kavtaradze 18 Mar '16

18 Mar '16

I am trying to write a C User Defined Function. It works in Neatbeans. What I want is, to take value from cells, insert them in text or char array and compare each corresponding array position to each other. As values I will have "1"s and "0s". E.g "101010100011". THe code I wrote in udf.c: /* Comparing bag of words */ char BitAnd(char Cell1[], char Cell2[]) { Cell1[10000]; Cell2[10000]; int counter; float result; int strlen = sizeof(Cell1)/sizeof(Cell1[0])-1; int i; for (i=0; i < strlen; i++) { if (Cell1[i] == Cell2[i]) { counter++; } } result = (float)counter / strlen; } /* MAL wrapper */ char UDFBitAnd(char Cell1[], char Cell2[]) { /* assert calling sanity */ assert(Cell1[] != NULL && Cell2[] != NULL); return MAL_SUCCEED; } In udf.mal I have: command BitAnd(Cell1:chr,Cell2:chr):flt address UDFBitAnd comment "Compare bag of words"; In 80_udf.sql: -- Comparing bag of words create function BitAnd(Cell1[] char, Cell2[] char) returns float external name udf.BitAnd; I compiled Monetdb, but when I try to connect with "monetdb" to db it gives me error. In the log file I found: MSG voc[18857]: !SyntaxException:parseError:command BitAnd(Cell1:chr,Cell2:chr):flt MSG voc[18857]: !SyntaxException:parseError: ^Type identifier expected MSG voc[18857]: !SyntaxException:parseError:command BitAnd(Cell1:chr,Cell2:chr):flt MSG voc[18857]: !SyntaxException:parseError: ^',' expected MSG merovingian[8780]: database 'voc' (18857) has exited with exit status 0 MSG merovingian[8780]: database 'voc' has shut down ERR merovingian[8780]: client error: database 'voc' started up, but failed to open up a communication channel

1 0

Adapting BATs
by Till Kolditz 18 Mar '16

18 Mar '16

Hi there, I'm just new to the list, so hello to everyone :-) I'm currently trying to find out how to adapt BATs in MonetDB for a scientific project. What I want to do is to add an encoding to data as some kind of a native DB feature. I.e. that a BAT stores data in some encoded fashion to detect certain kinds of errors that alter data in memory. By that, talking e.g. about an int BAT (column), the data width would increase e.g. from 4 to 5 Bytes (just as an example) where the additional byte stores the coding information. As I see it there are for a first shot two possibilities to do that: 1) Take one atom (e.g. int) and adapt all BAT / atom access code which touches int BATs. 2) Add some kind of new atom which is handled in a specific way. Since I have just started looking into MonetDB I now hope for a few hints to decide whether one of these approaches is sensible and whether there is maybe a simpler way. Thanks a lot in advance for any helpful comments :-) Best Till Kolditz -- Dipl.-Inf. Till Kolditz wiss. Mitarbeiter Technische Universität Dresden Fakultät Informatik Lehrstuhl Datenbanken 01062 Dresden Tel.: +49 (351) 463-38077 Fax: +49 (351) 463-38259 E-Mail: Till.Kolditz(a)tu-dresden.de jExam Group www: http://www.jexam.de Mail: till.kolditz(a)jexam.de

1 0

New Jun2016 stable branch created
by Sjoerd Mullender 14 Mar '16

14 Mar '16

I have created a new stable branch for the next feature release. This branch was created in order to stabilize the code before the feature release. From now on *only* bug fixes may go to this new stable branch. I reserve the right to undo any changes that do not comply with this rule. I also reserve the right to admit changes that are not strictly bug fixes. The name for the branch is Jun2016. So in each of your stable checkout directories you can do hg pull hg update -rJun2016 to update to this new branch. -- Sjoerd Mullender

1 0

Re: MonetDB: leftmart - trying to fix Windows compilation: do not "i...
by Lefteris 08 Mar '16

08 Mar '16

Stefan, this needs to be inlined because it is used many times. I will deal with compilation issues later on. This is a dev branch and we are running different experiments every night. But with this change if measured times are different I would not know if it is because of other changes or because of the inline statement removal. I would appreciate if you did not make any changes in this branch at the moment. thank you, lefteris On Mon, Mar 7, 2016 at 8:34 PM, Stefan Manegold < commits+Stefan.Manegold=cwi.nl(a)monetdb.org> wrote: > Changeset: ea1aed8beff8 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=ea1aed8beff8 > Modified Files: > gdk/gdk_bloom.c > Branch: leftmart > Log Message: > > trying to fix Windows compilation: do not "inline" BLOOMask() > > On Windows, compilation fails with > failed invocation: cl -GF -W3 -WX -MD -nologo -Ox -Zi -Oi -I. -I.\.. > -I.\..\..\gdk -I..\common\options -I.\..\..\gdk\..\common\options > -I..\common\stream -I.\..\..\gdk\..\common\stream -I..\common\utils > -I.\..\..\gdk\..\common\utils -I.\..\..\buildtools\conf -DHAVE_ICONV > -IC:\Libraries\iconv-1.11.1.win64-vs2010\include -DHAVE_LIBATOMIC_OPS > -IC:\Libraries\atomic_ops-7.4.2 -DNDEBUG -LD -Felibbat.dll gdk_batop.obj > gdk_select.obj gdk_search.obj gdk_hash.obj gdk_tm.obj gdk_orderidx.obj > gdk_align.obj gdk_bbp.obj gdk_heap.obj gdk_utils.obj gdk_atoms.obj > gdk_qsort.obj gdk_ssort.obj gdk_storage.obj gdk_bat.obj gdk_delta.obj > gdk_cross.obj gdk_system.obj gdk_value.obj gdk_posix.obj gdk_logger.obj > gdk_sample.obj gdk_calc.obj gdk_aggr.obj gdk_group.obj gdk_imprints.obj > gdk_bloom.obj gdk_join.obj gdk_project.obj gdk_unique.obj gdk_firstn.obj > libbat.res /link ..\common\options\monet_options.obj > ..\common\stream\libstream.lib ..\common\utils\mutils.obj > ..\common\utils\prompt.obj wsock32.lib Ws2_32.l > ib C:\Libraries\zlib-1.2.8.win64-vs2010\lib\zdll.lib > C:\Libraries\bzip2-1.0.6.win64-vs2010\lib\libbz2.lib psapi.lib > Creating library libbat.lib and object libbat.exp > gdk_join.obj : error LNK2019: unresolved external symbol BLOOMask > referenced in function hashjoin > libbat.dll : fatal error LNK1120: 1 unresolved externals > > > Maybe, the fact that BLOOMask() is "inline"d triggers this problem ... > > > diffs (11 lines): > > diff --git a/gdk/gdk_bloom.c b/gdk/gdk_bloom.c > --- a/gdk/gdk_bloom.c > +++ b/gdk/gdk_bloom.c > @@ -240,7 +240,6 @@ BATbloom(BAT *b) > return GDK_SUCCEED; > } > > -inline > int BLOOMask(BUN v, Bloomfilter *bloom) > { > BUN hv,mv,x,y,z; > _______________________________________________ > checkin-list mailing list > checkin-list(a)monetdb.org > https://www.monetdb.org/mailman/listinfo/checkin-list >

3 2

Re: MonetDB: leftmart - silence some compilers compaining about prin...
by Stefan Manegold 07 Mar '16

07 Mar '16

Hi Lefteris, the portable format string that we ussually use for size_t in MonetDB is "SZFMT" Best, Stefan ----- On Mar 7, 2016, at 2:38 PM, lsidir commits+lsidir=cwi.nl(a)monetdb.org wrote: > Changeset: a76605fb0b33 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=a76605fb0b33 > Modified Files: > gdk/gdk_bloom.c > Branch: leftmart > Log Message: > > silence some compilers compaining about printing a size_t as unsigned int or > long > > > diffs (12 lines): > > diff --git a/gdk/gdk_bloom.c b/gdk/gdk_bloom.c > --- a/gdk/gdk_bloom.c > +++ b/gdk/gdk_bloom.c > @@ -183,7 +183,7 @@ BATbloom(BAT *b) > > ALGODEBUG fprintf(stderr, "#BATbloom(b=%s#" BUNFMT ") %s: " > "create bloom filter: mbits = " BUNFMT ", ratio = " BUNFMT > - ", kfunc = %d, bytes = " BUNFMT "\n", BATgetId(b), > + ", kfunc = %d, bytes = %lu\n", BATgetId(b), > BATcount(b), b->T->heap.filename, > bloom->mbits, bloom->mbits/BATcount(b), > bloom->kfunc, bloom->bytes); > _______________________________________________ > checkin-list mailing list > checkin-list(a)monetdb.org > https://www.monetdb.org/mailman/listinfo/checkin-list -- | Stefan.Manegold(a)CWI.nl | DB Architectures (DA) | | www.CWI.nl/~manegold/ | Science Park 123 (L321) | | +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) |

1 0

Monetdb database crashes
by Souri Ratnaparkhi 04 Mar '16

04 Mar '16

Hello Team, Hello monetdb team, We have a table with 126 columns and approx. 94000 rows. We are trying to load approx 2500 records from temporary table into main table using query “insert into select” query. The database crashes upon query execution. No specific error is created in log file. When tried with 1500 records, it works. We also created another table with same DDL, that table is empty & inserted all the records from temp successfully without any issue. What could be the reason? Is 126 cols an issue? Please help. We are stuck. (btw: We have machine about 200GB of hard disk & 16 GB RAM (14 GB free)) Thank you [image: Description: Description: Description: Description: logo] * Souri Ratnaparkhi United Software Associates Pvt. Ltd. Pune, India Mob: 8975677255* * Phone (India Lines): (91) 020-25445550 **www.usain.com* <http://www.usain.com/>

2 1