- developers-list - monetdb.org

Can I expect to be able to use BBP code without GDKinit'ing?
by Eyal Rozenberg 24 Nov '16

24 Nov '16

Bottom line question: BBPinit() is not exposed in gdk_bbp.h, but rather only called from GDKinit() (which sees it through gdk_private.h). I just want to load data from persisted BAT files - not into MonetDB and not within the mserver5 process - so, speaking conceptually, I do not want to initialize the GDK, but do want to initialize the BBP. Questions: * Should BBPinit() work outside the scope of a GDKinit()? * If not, is it adaptable so as to allow this? * More generally, how much of GDK do I need to have running, just so as load persisted data into memory, using the code in gdk_bbp (or a slight variation thereof)? Now for the introduction and the motivation: As I mentioned a few MADADMs ago, I'll need to load persisted MonetDB columns into my GPU kernel testbench. This is relatively simple for numeric columns, given some scripting work to extract catalog data in a parsable format or to build named symlinks to columns (eg /path/to/dbfarm/tpch-sf-1/named_bats/lineitem/l_shipdate -> /path/to/dbfarm/tpch-sf-1/bat/12/34.tail). But it won't do for other kinds of data, most importantly strings - which I do want to work on. Plus, writing a wider-scope loader will let me avoid depending on MonetDB running for access to persisted columns. So, I'm writing a (selective) loader of persisted MonetDB columns, or a BBP loader if you will. My strategy is the following: 1. Copying the files in the GDK codebase which are necessary for building a binary which can call all code in gdk_bbp.h and not have unmet dependencies (this is mostly done). 2. Get that leaner slice of code, with a small main(), to actually work, i.e. not fail due to weird errors or result in junk data. 3. Peel away functions from the code I've copied from the repository which are not actually used. 4. Peel away the parts of the code which are not necessary for the actual loading - from within functions. In this initial stage this may involve work that becomes unnecessary when you have auxiliary data obtained from querying the DB (such as table-column name to BAT filename mapping). 5. Expand functionality and/or optimize performance after the peeling and/or C++ify for integration with my code 6. If other people / MDBS are interested, collaborate on making this support different BBP versions - so that the result is an inter-version export/forensics library.

2 1

Re: developers-list Digest, Vol 51, Issue 4
by Joweria Nakibuuka 23 Nov '16

23 Nov '16

the command i used: ./configure --prefix=/home/joweria/MonetDB-Nov2016-SP4 --enable-geom=yes --enable-pyintegration=yes --with-geos=yes and then after compilation under the LDFLAGS-enabled componets geom is one of the enabled componet 2016-11-10 13:00 GMT+02:00 <developers-list-request(a)monetdb.org>: > Send developers-list mailing list submissions to > developers-list(a)monetdb.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://www.monetdb.org/mailman/listinfo/developers-list > or, via email, send a message with subject or body 'help' to > developers-list-request(a)monetdb.org > > You can reach the person managing the list at > developers-list-owner(a)monetdb.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of developers-list digest..." > > > Today's Topics: > > 1. Need Help on enabling GIS (MonetDB/GIS) module in MonetDB > (Joweria Nakibuuka) > 2. Re: Need Help on enabling GIS (MonetDB/GIS) module in MonetDB > (Ying Zhang) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Wed, 9 Nov 2016 17:39:17 +0100 > From: Joweria Nakibuuka <joweria.nakibuuka(a)unitn.it> > To: developers-list(a)monetdb.org > Subject: Need Help on enabling GIS (MonetDB/GIS) module in MonetDB > Message-ID: > <CANbNUqZfu_pfzsZ=xUesfyLQPw3Ji3DW_fYeNOSZwfkTw= > 1Hsg(a)mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > Dear Admin, > I am running Ubuntu linux 14.04 LTS, x86_64-Ubuntu-linux-gnu/64bit with > 128bit integers. i recently installed MonetDB5 default branch from the > mercurial repository. > > I tried to rebuild and enable geom module but to no avail, it shows no > error, after compilation it shows that the module is enabled,However when i > run the mserver the monetDB/GIS module is not loaded as expected. I am > stuck and i don't seem to know how to proceed forward in order to run my > spatial queries > > Are there any setting i need to do explicitly to load this module? > > Thank you > > Regards, > Joweria nakibuuka >

2 2

Need Help on enabling GIS (MonetDB/GIS) module in MonetDB
by Joweria Nakibuuka 10 Nov '16

10 Nov '16

Dear Admin, I am running Ubuntu linux 14.04 LTS, x86_64-Ubuntu-linux-gnu/64bit with 128bit integers. i recently installed MonetDB5 default branch from the mercurial repository. I tried to rebuild and enable geom module but to no avail, it shows no error, after compilation it shows that the module is enabled,However when i run the mserver the monetDB/GIS module is not loaded as expected. I am stuck and i don't seem to know how to proceed forward in order to run my spatial queries Are there any setting i need to do explicitly to load this module? Thank you Regards, Joweria nakibuuka

2 1

Two new juicy assertions in the SQL front-end
by Dean De Leo 08 Nov '16

08 Nov '16

Dear MonetDB developers, I'm reporting two other assertions I hit in the front-end. The first is related to scalar subqueries, see query3.sql for a repro and backtrace3.txt. I guess it could be rather considered a feature request... BTW I appreciate this style of treating unsupported features, I'm starting to use in my changes as well :) The second occurs somewhere in the query rewriter when issuing the whole query, see query4.sql and backtrace4.txt. On Linux, tip of the default branch, debug build, empty tables, schema attached. Thanks, Dean

1 0

Two sweet crashes in the sql front-end
by Dean De Leo 08 Nov '16

08 Nov '16

Dear MonetDB developers, I would like to report a crash due to a null pointer using the attached schema.sql and query1a.sql, see backtrace1.txt. Eventually in the process of simplyfying the query to make this report, I stumbled upon a second assertion, see query2_crash_repro.sql and backtrace2.txt On Linux, tip of the default branch, debug build, empty tables. Dean

1 0

Re: MonetDB: malupgrade - Clear variables before resetting mb->vtop.
by Martin Kersten 07 Nov '16

07 Nov '16

WOW .... that took me quite some time nailing the example and looking into the wrong direction thx On 07/11/16 12:38, Sjoerd Mullender wrote: > Changeset: 8ac922886df2 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=8ac922886df2 > Modified Files: > monetdb5/mal/mal_instruction.c > Branch: malupgrade > Log Message: > > Clear variables before resetting mb->vtop. > > > diffs (20 lines): > > diff --git a/monetdb5/mal/mal_instruction.c b/monetdb5/mal/mal_instruction.c > --- a/monetdb5/mal/mal_instruction.c > +++ b/monetdb5/mal/mal_instruction.c > @@ -197,13 +197,13 @@ freeMalBlk(MalBlkPtr mb) > mb->stmt[i] = NULL; > } > mb->stop = 0; > + for(i=0; i< mb->vtop; i++) > + if (isVarConstant(mb, i)) > + VALclear(&getVarConstant(mb,i)); > mb->vtop = 0; > mb->vid = 0; > GDKfree(mb->stmt); > mb->stmt = 0; > - for(i=0; i< mb->vtop; i++) > - if (isVarConstant(mb, i)) > - VALclear(&getVarConstant(mb,i)); > GDKfree(mb->var); > mb->var = 0; > > _______________________________________________ > checkin-list mailing list > checkin-list(a)monetdb.org > https://www.monetdb.org/mailman/listinfo/checkin-list >

1 0

Data lost
by Luciano Sasso 17 Oct '16

17 Oct '16

Hi, I am delete rows from table, with "delete from mytable where id=x", and rows deleted no return in select "select * from mytable where id=x", its OK ! Version "Jul2015-SP1" But, when restart database, deleted rows return in select "select * from mytable where id=x". Its bug ?, or i have execute vacuum command, how to execute vacuum command. ? -- Luciano Sasso Vieira Data Scientist & Solutions Architect luciano(a)gsgroup.com.br <http://www.gsgroup.com.br> | tel: 17 3353-0833 | 17 3202-4040 | cel: 17 99706-9335 www.gsgroup.com.br <http://www.gsgroup.com.br> --- Este email foi escaneado pelo Avast antivírus. https://www.avast.com/antivirus

2 1

New Dec2016 stable branch created
by Sjoerd Mullender 14 Oct '16

14 Oct '16

I have created a new stable branch for the next feature release. This branch was created in order to stabilize the code before the feature release. From now on *only* bug fixes may go to this new stable branch. I reserve the right to undo any changes that do not comply with this rule. I also reserve the right to admit changes that are not strictly bug fixes. The name for the branch is Dec2016. So in each of your stable checkout directories you can do hg pull hg update -rDec2016 to update to this new branch. -- Sjoerd Mullender

1 0

Announcement: New Jun2016-SP2 Bugfix release of MonetDB suite
by Sjoerd Mullender 13 Oct '16

13 Oct '16

The MonetDB team at CWI/MonetDB BV is pleased to announce the Jun2016-SP2 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <http://www.monetdb.org/>. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Jun 2016-SP2 bugfix release Build Environment * We now use the CommonCrypto library instead of the OpenSSL library on Darwin. Bug Fixes * 3939: Assert failure on concurrent queries when querying sys.queue * 4019: Casting a timestamp from a string results in NULL * 4025: expressions in the WHERE clause that evaluates incorrectly * 4038: After upgrade from 11.21.19, jdbc couldn't list tables for non sys users * 4044: Server crash when trying to delete a table has been added to a merge table with "cascade" at the end * 4049: Wrong results for queries with "OR" and "LEFT JOIN" * 4052: Infinite loop in rel_select * 4054: copy into file wrongly exports functions * 4058: Server crashes with a particular conditional query * 4059: Geom functions only visible by user monetdb * 4060: BAT leak in some aggregate queries * 4062: Error: SELECT: no such binary operator 1800 790 9186'like(varchar,varchar)' when used in query running in other schema than sys * 4064: Assertion: column not found * 4067: Relevant column name not printed when a CSV parsing error occurs * 4070: Extra condition in join predicate of explicit join produces wrong MAL code * 4074: Cannot use prepared statements when caching disabled * 6065: CTE with row number and union fails within MAL

1 0

Bulk load from memory
by Hermano ... 09 Oct '16

09 Oct '16

Hello I became very interested in MonetDB during my research, and I am considering developing my own MAL compiler for a system based on a data model my research group is working on in the future. For now, I've been trying to understand MAL and how the COPY INTO command works in MonetDB Initially, I wish to do a large bulk load from an array in memory directly into MonetDB. My first "closed box" approach was to do the following: -Create a binary integer file containing milions of integers in /dev/shm/ named teste.bin. It is a memory based device, so the data is expected to be in main memory. -Create a table in MonetDB: CREATE TABLE teste (val INT); -Try to do BINARY COPY FROM: COPY BINARY INTO teste FROM ('/dev/shm/teste.bin'). When I try to this I get the error: Failed to attach file /dev/shm/teste.bin BATattach: cannot rename heapfile !OS: Invalid cross-device link After that I understood that monetdb basically moves the entire file around, which can't be done with data from another device. I wish to know if there is any sort of functionality in MonetDB or even a MAL module that would allow me to read from some sort of shared memory buffer (kind vague I know) directly into a BAT file. The MAL program for the COPY command has an sql.importTable and sql.append calls, so I wonder if there is some variant of those that would allow me to load binary data from a memory buffer? Thanks in Advance

1 0