The MonetDB team at CWI/MonetDB BV is pleased to announce the Nov2019 feature release of the MonetDB suite of programs. More information about MonetDB can be found on our website at https://www.monetdb.org/. For details on this release, please see the release notes at https://www.monetdb.org/Downloads/ReleaseNotes. As usual, the download location is https://dev.monetdb.org/downloads/. Nov 2019 feature release (11.35.3) MonetDB5 Server * The server "console" has been removed, as has the --daemon option. The server now doesn't read from standard input anymore. The way to stop a server is by sending it a TERM signal (on Linux/Unix) or by sending it an interrupt signal (usually control-C -- on all systems). * Implemented a function bat.diffcand to calculate difference of two candidate lists. * The mtime module was completely rewritten, the atom types date, daytime, and timestamp were changed. Upgrade code for BATs containing these types has been implemented. The old daytime type used a 32 bit integer to record milliseconds since the start of the day. The new daytime type uses a 64 bit integer to record microseconds since the start of the day. The old date type recorded days since or before the year 1. The new daytime type records the day of the month and the number of months since the year -4712 separately in a single 32 bit integer of which only 26 bits are used. Dates now use the proleptic Gregorian calendar, meaning the normal Gregorian callendar has been extended backward, and before the year 1, we have the year 0. Both the old and new timestamp types are a combination of a daytime and a date value, but since those types have changed, the timestamp type has also changed. The new date type has a smaller range than the old. The new date range is from the year -4712 to the year 170049. During conversion of date and timestamp columns, the dates are clamped to this range. * The tzone and rule atom types have been removed. They were not used by any code, and they were defined in a non-portable way. * Added "mapi_ipv6" property to monet_options to force ipv6 address binding only. This property is inherited while forking from monetdbd if it is also set there. * Removed (bat)calc.between_symmetric and changed (bat)calc.between by adding a number of extra arguments, all of type :bit: symmetric, low inclusive, high inclusive, nils false. Merovingian * Added "vmmaxsize" and "memmaxsize" mserver5 options to the daemon in order to set mserver5's maximum virtual and committed memory respectively. * Added ipv6 property to monetdbd properties to force IPv6 addresses binding only. By default this property is false to allow IPv4 addresses as well. MonetDB Common * BATcalcbetween and all its variants now have an extra bool parameter "anti" to invert the test. * All forms of BATcalcbetween and VARcalcbetween have extra bool arguments for low inclusive, high inclusive and nils false. The latter causes the result to be false instead of nil if the value being checked is nil. SQL Frontend * Removed functions json.text(string) returns string and json.text(int) returns string. Their MAL implementation didn't exist, so they were meaningless. * There are new aggregate functions sys.median_avg and sys.quantile_avg that return the interpolated value if the median/quantile doesn't fall exactly on a particular row. These functions always return a value of type DOUBLE and only work for numeric types (various width integers and floating point). * Added sys.deltas ("schema" string, "table" string, "column" string) returns table ("values" bigint) system function which returns a single column with 6 values: a flag indicating if the column's upper table is cleared or not, the count of the RDONLY, RD_INS and RD_UPD_ID deltas of the column itself, the number of deleted values of the column's table, as well as the level of the current transaction in the transaction level tree. It should be used for debugging purposes only. * Added "VALUES row_list" statement as a top SQL projection statement. * The implementation of in-expression now follows a join-based approach instead of using iterative union/selects. This greatly improves performance for large in-value-lists. Furthermore the old approach has large in-value-lists generate large MAL plans. This is now no longer the case. * Strings are now limited to 1GB, double-quoted tokens are limited to 2kB. These sizes are bytes of (UTF-8 encoded) input data. Bug Fixes * 3533: SQL aggregate functions avg(), sum() and median() return an error when used on a column with datatype interval second * 6134: Query produces error: HEAPalloc: Insufficient space for HEAP of 1168033427456 bytes. * 6613: LATERAL crash /.../rel_bin.c:1473: rel2bin_table: Assertion `0' failed. * 6683: Bug in subselect * 6686: Bug in subselect (count function) * 6688: Bug in subselect (or condition) * 6689: Trying to improve the performance of SQL queries with a large list of members in IN clause. * 6695: timestamp transformation * 6700: Monetdb Bugs in Subselect statements: * 6722: window functions issues * 6740: while upgrading the database from verison (MonetDB-11.27.13) to (MonetDB-11.33.3) we are unable to bring up the database * 6754: in mclient a strang msg is reported after issueing command: set schema sys; * 6755: Assertion failure in rel_bin.c * 6756: Error in optimizer garbageCollector on merge tables select * 6757: Double free or corruption (out) * 6758: SIGSEGV in __strcmp_sse2_unaligned() * 6759: Python JSON loader creates invalid data type for strings * 6761: Error: Program contains errors.:(NONE).multiplex * 6762: mserver5 crashes on (re-)start * 6764: mserver5 crashes with corruption, double free, invalid size or invalid pointer * 6766: Missing bulk implementation for get_value and next_value calls * 6769: ProfilerStart is not threadsafe * 6771: R-devel * 6772: TRACE includes information from all active sessions * 6773: json.filter returns corrupted string when selecting JSON null value * 6774: PROD aggregation gives wrong result * 6775: NOT IN with an AND containing an OR gives wrong result * 6776: Creating a table with a full outer join query gives type with wrong digits on the joined key. * 6779: Using Windows Messages translation for errno error codes. * 6780: Wrong value of the rank function * 6781: Insert after index creation crash * 6783: AVG changes scale of its results * 6784: function sys.isauuid(string) should return false if string value cannot be converted to a UUID