- developers-list - monetdb.org

Announcement: New Jun2016-SP2 Bugfix release of MonetDB suite
by Sjoerd Mullender 13 Oct '16

13 Oct '16

The MonetDB team at CWI/MonetDB BV is pleased to announce the Jun2016-SP2 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <http://www.monetdb.org/>. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Jun 2016-SP2 bugfix release Build Environment * We now use the CommonCrypto library instead of the OpenSSL library on Darwin. Bug Fixes * 3939: Assert failure on concurrent queries when querying sys.queue * 4019: Casting a timestamp from a string results in NULL * 4025: expressions in the WHERE clause that evaluates incorrectly * 4038: After upgrade from 11.21.19, jdbc couldn't list tables for non sys users * 4044: Server crash when trying to delete a table has been added to a merge table with "cascade" at the end * 4049: Wrong results for queries with "OR" and "LEFT JOIN" * 4052: Infinite loop in rel_select * 4054: copy into file wrongly exports functions * 4058: Server crashes with a particular conditional query * 4059: Geom functions only visible by user monetdb * 4060: BAT leak in some aggregate queries * 4062: Error: SELECT: no such binary operator 1800 790 9186'like(varchar,varchar)' when used in query running in other schema than sys * 4064: Assertion: column not found * 4067: Relevant column name not printed when a CSV parsing error occurs * 4070: Extra condition in join predicate of explicit join produces wrong MAL code * 4074: Cannot use prepared statements when caching disabled * 6065: CTE with row number and union fails within MAL

1 0

Bulk load from memory
by Hermano ... 09 Oct '16

09 Oct '16

Hello I became very interested in MonetDB during my research, and I am considering developing my own MAL compiler for a system based on a data model my research group is working on in the future. For now, I've been trying to understand MAL and how the COPY INTO command works in MonetDB Initially, I wish to do a large bulk load from an array in memory directly into MonetDB. My first "closed box" approach was to do the following: -Create a binary integer file containing milions of integers in /dev/shm/ named teste.bin. It is a memory based device, so the data is expected to be in main memory. -Create a table in MonetDB: CREATE TABLE teste (val INT); -Try to do BINARY COPY FROM: COPY BINARY INTO teste FROM ('/dev/shm/teste.bin'). When I try to this I get the error: Failed to attach file /dev/shm/teste.bin BATattach: cannot rename heapfile !OS: Invalid cross-device link After that I understood that monetdb basically moves the entire file around, which can't be done with data from another device. I wish to know if there is any sort of functionality in MonetDB or even a MAL module that would allow me to read from some sort of shared memory buffer (kind vague I know) directly into a BAT file. The MAL program for the COPY command has an sql.importTable and sql.append calls, so I wonder if there is some variant of those that would allow me to load binary data from a memory buffer? Thanks in Advance

1 0

Fwd: lib_pyapi complains about undefined symbol PyInt_Type
by Dean De Leo 06 Oct '16

06 Oct '16

Hi, on my personal machine, when starting mserver5, monetdb complains with the following message: !SyntaxException:parseError: !LoaderException:loadLibrary:Loading error failed to open library pyapi (from within file '/linux/workspace/monetdb/build/lib/monetdb5/lib_pyapi.so'): /linux/workspace/monetdb/build/lib/monetdb5/lib_pyapi.so: undefined symbol: PyInt_Type !SyntaxException:parseError: !<identifier> expected I guess this is another case of conflict Python 2 vs 3. I am using ArchLinux where the default is v3. In my case a workaround seems to be forcing back python2 (see patch). Dean

2 1

GDK_mmap_minsize again
by Hannes Mühleisen 24 Sep '16

24 Sep '16

Hello list, we were wondering about the purpose of GDK_mmap_minsize when creating transient columns. The attached patch will always *try* to malloc/realloc a transient column but still fall back to memory-mapped files if malloc should fail. This dramatically improves performance. Any good reason why this should not be the default behaviour? Thanks, Mark and Hannes

5 5

eviction in Monetdb
by Narendra Pradeep 15 Sep '16

15 Sep '16

Hello , Need to know about monetdb eviction policy ? My understanding is Monetdb creates a buffer pool {BBP} and inserts all the bats that are used . But not sure when does Monetdb deletes the bat from the buffer pool and what eviction policy does it use (like LRU or based on some threshold) ? Thanks

2 1

Re: MonetDB: graph1 - Add the operator bat.getCount() to retrieve th...
by Stefan Manegold 14 Sep '16

14 Sep '16

Hi Dean, how is bat.getCount() different from aggr.count() ? ;-) Thanks! Stefan ----- On Sep 14, 2016, at 6:30 PM, Dean De Leo commits+dleo=cwi.nl(a)monetdb.org wrote: > Changeset: 9ac831deacb3 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=9ac831deacb3 > Modified Files: > monetdb5/modules/kernel/bat5.c > monetdb5/modules/kernel/bat5.h > monetdb5/modules/kernel/bat5.mal > Branch: graph1 > Log Message: > > Add the operator bat.getCount() to retrieve the cardinality of a bat > > > diffs (48 lines): > > diff --git a/monetdb5/modules/kernel/bat5.c b/monetdb5/modules/kernel/bat5.c > --- a/monetdb5/modules/kernel/bat5.c > +++ b/monetdb5/modules/kernel/bat5.c > @@ -496,6 +496,19 @@ BKCgetColumnType(str *res, const bat *bi > } > > str > +BKCgetCount(lng *res, const bat *bid) > +{ > + *res = lng_nil; > + if (BBPcheck(*bid, "bat.getCount")) { > + BAT *b = BBPquickdesc(*bid, 0); > + > + if (b != NULL) > + *res = (lng) BATcount(b); > + } > + return MAL_SUCCEED; > +} > + > +str > BKCgetRole(str *res, const bat *bid) > { > BAT *b; > diff --git a/monetdb5/modules/kernel/bat5.h b/monetdb5/modules/kernel/bat5.h > --- a/monetdb5/modules/kernel/bat5.h > +++ b/monetdb5/modules/kernel/bat5.h > @@ -29,6 +29,7 @@ mal_export str BKCbun_inplace_force(bat > mal_export str BKCbat_inplace_force(bat *r, const bat *bid, const bat *rid, > const bat *uid, const bit *force); > mal_export str BKCgetCapacity(lng *res, const bat *bid); > mal_export str BKCgetColumnType(str *res, const bat *bid); > +mal_export str BKCgetCount(lng *res, const bat *bid); > mal_export str BKCgetRole(str *res, const bat *bid); > mal_export str BKCsetkey(bat *res, const bat *bid, const bit *param); > mal_export str BKCisSorted(bit *res, const bat *bid); > diff --git a/monetdb5/modules/kernel/bat5.mal b/monetdb5/modules/kernel/bat5.mal > --- a/monetdb5/modules/kernel/bat5.mal > +++ b/monetdb5/modules/kernel/bat5.mal > @@ -77,6 +77,10 @@ command getCapacity(b:bat[:any_1]):lng > address BKCgetCapacity > comment "Returns the current allocation size (in max number of elements) of a > BAT."; > > +command getCount(b:bat[:any_1]):lng > +address BKCgetCount > +comment "Returns the current tuple count of a BAT."; > + > command getColumnType( b:bat[:any_1] ) :str > address BKCgetColumnType > comment "Returns the type of the tail column of a BAT, as an integer type > number."; > _______________________________________________ > checkin-list mailing list > checkin-list(a)monetdb.org > https://www.monetdb.org/mailman/listinfo/checkin-list -- | Stefan.Manegold(a)CWI.nl | DB Architectures (DA) | | www.CWI.nl/~manegold/ | Science Park 123 (L321) | | +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) |

1 0

BAT in memory
by Narendra Pradeep 30 Aug '16

30 Aug '16

Hello , I am reading monetdb source code . I need to know which function check whether the bat of the particular column lies in memory or not ? And how does it maps the bat with corresponding column ? Thanks . Regars , Naren

1 0

Monetdb printing values of variables in console or terminal
by Narendra Pradeep 23 Aug '16

23 Aug '16

Hai everyone , How to print messages in console or to write something in log file of the Monetdb . can someone help me with exact function ? I cant even write a word in logfile and console is there something with my build ? Regards , Pradeep

1 0

compression in monetdb
by Narendra Pradeep 11 Aug '16

11 Aug '16

Hai , Read that dictionary encoding occurs in monetdb . so 1) what is dictionary compression technique ? 2) Does it works for strings alone or other data structures too ? 3) I a mailing list it was stated that it works only till the size of the column in 128 MB for older release? Is this the same for current release ? 4) Does Monetdb uses any other compression technique ? 5) If yes where can I find the source of it (Precisely , which file in source ) Thanks in advance regards , Pradeep

1 0

Parameters in bind function
by Narendra Pradeep 08 Aug '16

08 Aug '16

Hello everyone , I am trying to understand the MAL statements for joining three tables . On which I came across bind function in MAL statement . I have a table lineitem with l_quantity in it . So what does the following statements means 1 ) ( X_94:bat[:oid,:oid] ,X_95:bat[:oid,:int] ) := sql.bind(X_4,"sys","lineitem","l_quantity",2,0,4); 2 ) (X_96:bat[:oid,:oid] ,X_97:bat[:oid,:int] ) := sql.bind(X_4,"sys","lineitem","l_quantity",2,1,4); what does the 5 th and 6 th argument means ? Regards , Pradeep

1 0