- users-list - monetdb.org

Announcement: New Nov2019-SP1 Bugfix release of MonetDB suite
by Sjoerd Mullender 20 Dec '19

20 Dec '19

The MonetDB team at CWI/MonetDB BV is pleased to announce the Nov2019-SP1 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <https://www.monetdb.org/>. For details on this release, please see the release notes at <https://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <https://dev.monetdb.org/downloads/>. Nov 2019-SP1 bugfix release (11.35.9) Bug Fixes * 6723: columns aliases duplicates should not be allowed. automatic aliasing required. * 6724: Prepare confuses types when more than one argument is used * 6726: Python aggregation does not create aggr_group when aggregating over all rows * 6765: GRANT SELECT privilege on a subset of table columns results in access denied error when selecting the same columns from the table * 6790: Count distinct giving wrong results * 6791: str_to_time('11:40', '%H:%M') creates wrong time value * 6792: JSON path compiler accepts invalid input * 6793: cast(interval second value to int or decimal) is wrong (by a factor of 1000), cast(interval month value to decimal or floating point) fails * 6794: external name fits.listdir not bound (sys.listdir) Fatal error during initialization: * 6796: Incorrect crash time reported by monetdb tool after crash of mserver5 * 6798: json.text off by one error * 6804: DNS resolution of 0.0.0.0 fails on recent Ubuntus

1 0

resetting credentials from mserver5?
by Jonathan Speiser 20 Dec '19

20 Dec '19

Hello, I had tried changing the password on my monetdb server (MonetDB 5 server v11.29.3 "Mar2018") but after rebooting it seems I am locked out from mclient. Is there any way I can change the monetdb user password from mserver5? Thank you for your help Jon

2 1

starting database 'tpch'... FAILED
by DEMBELE Simon Pierre 16 Dec '19

16 Dec '19

-------- Original Message -------- Subject: starting database 'tpch'... FAILED Date: 16/12/2019 12:58 From: DEMBELE Simon Pierre <simon-pierre.dembele(a)ensma.fr> To: mailman-owner(a)monetdb.org Bonjour, Hello I need help; Why can solve this issue starting database 'tpch'... FAILED start: starting 'tpch' failed: database 'tpch' has crashed after starting, manual intervention needed, check monetdbd's logfile (merovingian.log) for details Thank -- Simon

2 2

MERGE TABLE - CANNOT DROP PARTITION
by Ariel Abadi 10 Dec '19

10 Dec '19

Hi! I'm still having issue dropping internal tables from the MERGE TABLE. I used to have this problem on a previous version. I have created a new DB, imported all the data... and now I need to delete a partition (NOT a table). I'm still facing the same problem, which is that I cannot remove it from the merge table. (as you can see below).. Is there anything I can do? Rgds Ariel Welcome to mclient, the MonetDB/SQL interactive terminal (Apr2019) Database: MonetDB v11.33.3 (Apr2019), 'mapi:monetdb://dt-mn:54321/cli_dg' Type \q to quit, \? for a list of available commands auto commit mode: on sql>ALTER TABLE sb_traf.cub15_cubo DROP TABLE sb_traf.cub8_20190801; operation successful sql>ALTER TABLE sb_traf.cub15_cubo DROP TABLE sb_traf.cub8_20190801; operation successful sql>ALTER TABLE sb_traf.cub15_cubo DROP TABLE sb_traf.cub8_20190801; operation successful sql>ALTER TABLE sb_traf.cub15_cubo DROP TABLE sb_traf.cub8_20190801; operation successful sql>ALTER TABLE sb_traf.cub15_cubo DROP TABLE sb_traf.cub8_20190801; operation successful sql>ALTER TABLE sb_traf.cub15_cubo DROP TABLE sb_traf.cub8_20190801; operation successful sql>ALTER TABLE sb_traf.cub15_cubo DROP TABLE sb_traf.cub8_20190801; operation successful sql>ALTER TABLE sb_traf.cub15_cubo DROP TABLE sb_traf.cub8_20190801; operation successful

2 1

CREATE DATABASE
by Ariel Abadi 09 Dec '19

09 Dec '19

Hi! Im trying to start a new fresh database .. I am creating with the following commands: monetdbd create /opt/starbi/mm/ monetdbd start /opt/starbi/mm/ monetdb create cli_dg monetdb start cli_dg When I try to start the new database... the database is NOT starting. starting database 'cli_dg'... FAILED start: starting 'cli_dg' failed: database 'cli_dg' has crashed after starting, manual intervention needed, check monetdbd's logfile (merovingian.log) for details *Here is the log:* 2019-12-09 15:59:07 MSG cli_dg[385]: # loading sql script: 72_fits.sql 2019-12-09 15:59:07 ERR cli_dg[385]: #main thread:!ERROR:ParseException:SQLparser:3F000!CREATE PROCEDURE: external name fits.listdir not bound (sys.listdir) 2019-12-09 15:59:07 ERR cli_dg[385]: ParseException:SQLparser:3F000!CREATE PROCEDURE: external name fits.listdir not bound (sys.listdir)Fatal error during initialization: 2019-12-09 15:59:07 ERR cli_dg[385]: ParseException:SQLparser:3F000!CREATE PROCEDURE: external name fits.listdir not bound (sys.listdir) 2019-12-09 15:59:11 MSG merovingian[370]: database 'cli_dg' (385) has exited with exit status 1 2019-12-09 15:59:11 ERR control[370]: (local): failed to fork mserver: database 'cli_dg' has crashed after starting, manual intervention needed, check monetdbd's logfile (merovingian.log) for details Can you please help!?

4 5

COPY INTO ... LOCKED
by Roberto Cornacchia 02 Dec '19

02 Dec '19

Hi, The documentation about the LOCKED modifier for COPY INTO mentions: *When using the LOCKED mode, it is the responsibility of the database user to make sure that there is currently only a single client connection accessing the database* I was assuming (and the syntax suggests, I think) that the LOCKED refers to the table being copied into, not to the entire database. That is, only that table is updated without the overhead of logging. But the documentation imposes the very restrictive condition that only one connection is active on the database. That seems to suggest that the entire database is being updated without logging, not only that table. Could you please clarify? Can I do COPY INTO LOCKED safely with more than one connection, as long as only 1 connection touches that table? Roberto

1 0

Issue in using Monet/Python loader
by Kalaiarasi Gowtham 29 Nov '19

29 Nov '19

Hi, While trying to use Monet/Python Loader function,we are getting the following error *pymonetdb.exceptions.OperationalError: 'pyapi.eval_loader' undefined in: pyapi.eval_loader(0x7f34f01f2c60:ptr, "{_emit.emit( { 'event_date': '2019-10-10', 'status': 1})};":str);* *Followed the below steps for installation* *Python installation steps:* We have installed python 3.6 1)yum install autoconf 2)yum install automake 3)yum install libtool 4)yum install openssl 5)yum install openssl-devel 6)yum install python-devel 7)Configured environment path settings for python export PYTHONPATH=$PATH:/usr/local/lib/python3.6/site-packages *Monet installation steps:* 1)git clone https://github.com/MonetDB/MonetDB.git 2)cd /MonetDB 3)./bootstrap 4)cd.. 5)mkdir testdir 6)cd testdir 7)../MonetDB/configure --enable-pyintegration=yes While configuring we get the following *py3integration is enabled* 8)make 9)make install 10)monetdbd create /path/to/mydbfarm 11)monetdbd start /path/to/mydbfarm 12)monet stop -a 13)monet set embedpy=true 14)monet start -a 13)Enabled numpy(pip install numpy) *Created a loader by using the below python script* import pymonetdb import sys import os connection = pymonetdb.connect(username="xyz", password="xyz123", hostname="ipaddress", database="testdb") cursor = connection.cursor() cursor.execute("CREATE LOADER myloader() LANGUAGE PYTHON {_emit.emit( { 'event_date': '2019-10-10', 'status': 1})};") #create loader cursor.execute("COPY LOADER INTO store FROM myloader();") #append the row from loader to table connection.commit() *While running the above script getting the following error* pymonetdb.exceptions.OperationalError: 'pyapi.eval_loader' undefined in: pyapi.eval_loader(0x7f34f01f2c60:ptr, "{_emit.emit( { 'event_date': '2019-10-10', 'status':1})};":str); Please, help us to fix this issue. Thanks & Regards Kalaiarasi A

2 1

Announcement: New Nov2019 Feature release of MonetDB suite
by Sjoerd Mullender 28 Nov '19

28 Nov '19

The MonetDB team at CWI/MonetDB BV is pleased to announce the Nov2019 feature release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <https://www.monetdb.org/>. For details on this release, please see the release notes at <https://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <https://dev.monetdb.org/downloads/>. Nov 2019 feature release (11.35.3) MonetDB5 Server * The server "console" has been removed, as has the --daemon option. The server now doesn't read from standard input anymore. The way to stop a server is by sending it a TERM signal (on Linux/Unix) or by sending it an interrupt signal (usually control-C -- on all systems). * Implemented a function bat.diffcand to calculate difference of two candidate lists. * The mtime module was completely rewritten, the atom types date, daytime, and timestamp were changed. Upgrade code for BATs containing these types has been implemented. The old daytime type used a 32 bit integer to record milliseconds since the start of the day. The new daytime type uses a 64 bit integer to record microseconds since the start of the day. The old date type recorded days since or before the year 1. The new daytime type records the day of the month and the number of months since the year -4712 separately in a single 32 bit integer of which only 26 bits are used. Dates now use the proleptic Gregorian calendar, meaning the normal Gregorian callendar has been extended backward, and before the year 1, we have the year 0. Both the old and new timestamp types are a combination of a daytime and a date value, but since those types have changed, the timestamp type has also changed. The new date type has a smaller range than the old. The new date range is from the year -4712 to the year 170049. During conversion of date and timestamp columns, the dates are clamped to this range. * The tzone and rule atom types have been removed. They were not used by any code, and they were defined in a non-portable way. * Added "mapi_ipv6" property to monet_options to force ipv6 address binding only. This property is inherited while forking from monetdbd if it is also set there. * Removed (bat)calc.between_symmetric and changed (bat)calc.between by adding a number of extra arguments, all of type :bit: symmetric, low inclusive, high inclusive, nils false. Merovingian * Added "vmmaxsize" and "memmaxsize" mserver5 options to the daemon in order to set mserver5's maximum virtual and committed memory respectively. * Added ipv6 property to monetdbd properties to force IPv6 addresses binding only. By default this property is false to allow IPv4 addresses as well. MonetDB Common * BATcalcbetween and all its variants now have an extra bool parameter "anti" to invert the test. * All forms of BATcalcbetween and VARcalcbetween have extra bool arguments for low inclusive, high inclusive and nils false. The latter causes the result to be false instead of nil if the value being checked is nil. SQL Frontend * Removed functions json.text(string) returns string and json.text(int) returns string. Their MAL implementation didn't exist, so they were meaningless. * There are new aggregate functions sys.median_avg and sys.quantile_avg that return the interpolated value if the median/quantile doesn't fall exactly on a particular row. These functions always return a value of type DOUBLE and only work for numeric types (various width integers and floating point). * Added sys.deltas ("schema" string, "table" string, "column" string) returns table ("values" bigint) system function which returns a single column with 6 values: a flag indicating if the column's upper table is cleared or not, the count of the RDONLY, RD_INS and RD_UPD_ID deltas of the column itself, the number of deleted values of the column's table, as well as the level of the current transaction in the transaction level tree. It should be used for debugging purposes only. * Added "VALUES row_list" statement as a top SQL projection statement. * The implementation of in-expression now follows a join-based approach instead of using iterative union/selects. This greatly improves performance for large in-value-lists. Furthermore the old approach has large in-value-lists generate large MAL plans. This is now no longer the case. * Strings are now limited to 1GB, double-quoted tokens are limited to 2kB. These sizes are bytes of (UTF-8 encoded) input data. Bug Fixes * 3533: SQL aggregate functions avg(), sum() and median() return an error when used on a column with datatype interval second * 6134: Query produces error: HEAPalloc: Insufficient space for HEAP of 1168033427456 bytes. * 6613: LATERAL crash /.../rel_bin.c:1473: rel2bin_table: Assertion `0' failed. * 6683: Bug in subselect * 6686: Bug in subselect (count function) * 6688: Bug in subselect (or condition) * 6689: Trying to improve the performance of SQL queries with a large list of members in IN clause. * 6695: timestamp transformation * 6700: Monetdb Bugs in Subselect statements: * 6722: window functions issues * 6740: while upgrading the database from verison (MonetDB-11.27.13) to (MonetDB-11.33.3) we are unable to bring up the database * 6754: in mclient a strang msg is reported after issueing command: set schema sys; * 6755: Assertion failure in rel_bin.c * 6756: Error in optimizer garbageCollector on merge tables select * 6757: Double free or corruption (out) * 6758: SIGSEGV in __strcmp_sse2_unaligned() * 6759: Python JSON loader creates invalid data type for strings * 6761: Error: Program contains errors.:(NONE).multiplex * 6762: mserver5 crashes on (re-)start * 6764: mserver5 crashes with corruption, double free, invalid size or invalid pointer * 6766: Missing bulk implementation for get_value and next_value calls * 6769: ProfilerStart is not threadsafe * 6771: R-devel * 6772: TRACE includes information from all active sessions * 6773: json.filter returns corrupted string when selecting JSON null value * 6774: PROD aggregation gives wrong result * 6775: NOT IN with an AND containing an OR gives wrong result * 6776: Creating a table with a full outer join query gives type with wrong digits on the joined key. * 6779: Using Windows Messages translation for errno error codes. * 6780: Wrong value of the rank function * 6781: Insert after index creation crash * 6783: AVG changes scale of its results * 6784: function sys.isauuid(string) should return false if string value cannot be converted to a UUID

1 0

UDF bulk implementation with some arguments as scalars
by Roberto Cornacchia 05 Nov '19

05 Nov '19

Hi, When defining a UDF, e.g. CREATE FUNCTION myfunc(a int, b int) RETURNS int we can have: - a scalar implementation command myfunc(a:int, b:int) :int - a bulk implementation command myfunc(a:bat[:int], b:bat[:int]) :bat[:int] - a mixed implementation (e.g. a is a column, b is scalar): command myfunc(a:bat[:int], b:int) :bat[:int] I'm trying to understand what the restrictions are, if any, for the third case. Is it expected to be allowed in (a combination of) the following cases? - Interleaved bulk/scalar arguments command myfunc(a:bat[:int], b:int, c:bat[:int]) :bat[:int] - Table-returning function command myfunc(a:bat[:int], b:int) (:bat[:int], :bat[:int]) - Table-returning function, used with a sub-select in input SELECT * FROM myfunc( (SELECT a, 10 FROM mytable) ); In particular I can't get the last one to work. Even though the sub-select has a constant as the second column, this constant seems to be first replicated into a full column and then given to myfunc() as a column. When the constant parameters are a few, that becomes a waste of time/space. Plus the annoyance of selecting the first value of a constant column for the actual internal implementation, which of course expects a scalar for those arguments. Am I missing something? Is there a way to make that work as I would expect? Thanks, Roberto

4 5

Report a bug
by lhy1149 04 Nov '19

04 Nov '19

Hello, I don't know if it's appropriate, but because I can't register monetdb's bugzilla, I'm going to report a bug here. This problem causes the insert statement to hang and mserver5 takes up 100% of the CPU time. Here is the code that caused the problem: Create table a(a int, b int, id bigserial); -- comment any of the next 2 lines will resolve the problem Create ordered index a_pk on a(id); Create index a_idx1 on a(a); Insert into a(a) values(1); -- this line will hang Enviroment: macOS Catalina 10.15 MonetDB v11.33.11 I hope the development team can check if there is this problem. Thank you

3 2