Jul2017 (11.27)

These release notes are replicated for historical purposes only. The release is not supported anymore.

Jul 2017-SP4 bugfix release (11.27.13)

Build Environment

Added the .lib and .h files needed for building extensions to the Windows installer.

Bug Fixes

3470: Support setClob without length restrictions
6468: JDBC 2.27 fails with year < 1000
6482: Query failures on order by on union
6483: Monetdb crashes, on query
6487: UNION of NULLs with several tables fails
6488: Semijoin returns duplicate values from a column with unique values
6489: Sqlitelogictest - Wrong result set of complex conditional query
6490: Sqlitelogictest - Select query with an IN clause parse error
6491: SELECT IN returns NULL instead of false when not found
6492: Persistent hashes stored and then ignored. Storage info not in sync with actual indices.
6493: Sqlitelogictest - Aggregation query on empty table with DISTINCT clause
6494: Sqlitelogictest - Algebra operators priority in select query
6495: Sqlitelogictest - Omitting AS in a result set column alias name
6496: Sqlitelogictest - Select interval comparisons between floating-points and NULL
6497: Sqlitelogictest - Select not between query producing wrong results
6498: large virtual memory spike on BLOB column COUNT
6499: Crash when trying to replace a function defined in sys from a different schema
6502: Query with multiple limit clauses does not return anything
6508: Segmentation fault in mserver5 on Python2 UDF with TIMESTAMP column input that has NULL values (conversion.c:438, PyNullMask_FromBAT)
6510: Sqlitelogictest: Wrong output in aggregation query
6512: Monetdb crashes on query with limit after sort with case
6514: Sqlitelogictest: Range query between NULL values not possible
6515: Insert null second interval value results in 0
6516: Sqlitelogictest unknown bat append operation
6517: Sqlitelogictest overflow in conversion during MAL plan execution
6518: Sqlitelogictest: count aggregation with not in operator
6519: Sqlitelogictest: algebra join between lng and int BATs undefined
6520: UPDATE with correlated subquery causes assertion (or segfault)
6522: Sqlitelogictest: IN operator return a single column
6523: Sqlitelogictest: Case statement subquery missing
6524: Sqlitelogictest: Crash in aggregation query with IN operator
6527: Crash using order by alias in subquery

Jul 2017-SP3 bugfix release (11.27.11)

MonetDB Common

Reimplemented summing of a column of floating point (flt and dbl) values. The old code could give wildly inaccurate results when adding up lots and lots of values due to lack of precision. Try SELECT sum(c) FROM t; where t is 100,000,000 rows, c is of type REAL and all values are equal to 1.1. (The old code returned 33554432 instead of 1.1e8.)

Bug Fixes

3898: Deadlock on insertion
6429: ROUND produces wrong data type
6436: Query sequence with 2x ifthenelse() and next nullif() causes mserver5 Segmentation fault
6437: System schemas “profiler” and “json” shouldn’t be allowed to be dropped.
6439: Invalid references to sys.columns.id from sys.statistics.column_id
6442: SEGFAULT with COPY INTO BEST EFFORT and skipping input columns
6443: complex(?) query forgets(?) column name
6444: Using ‘with’ keyword with table returning function crashes monetdb
6445: Sqlitelogictest crash in MySQL query
6446: sql_parser.y bug?
6448: ‘insert into’ with multiple rows containing subqueries crashes
6449: Assertion error in rel_dce_refs (sqlsmith)
6450: Assertion error in exp_bin (sqlsmith)
6451: Assertion error in sql_ref_dec (sqlsmith)
6453: Assertion error in rel_rename_exps (sqlsmith)
6454: SQL lexical error
6455: Assertion error in rel_apply_rewrite (sqlsmith)
6456: NULL becomes 0 in outer join
6459: Assertion error in exp_bin (sqlsmith)
6462: large virtual memory spike on BLOB column select
6465: appending to variables sized atom bats other than str bats with force flag may result in corrupted heap
6467: date_to_str formatter is wrong
6470: mitosis gets in the way of simple select
6471: calls to sys.generate_series should auto-convert arguments
6472: Assertion failure in rel_rename (Sqlsmith)
6477: assertion eror rel_push_project_up (sqlsmith)
6478: Crash with nested order by/ limit offset
6479: Mserver receives an assertion error on a procedure call
6480: Segfault in mvc_find_subexp (sqlsmith)

Jul 2017-SP2 bugfix release (11.27.9)

MonetDB Common

A serious bug, possibly resulting in database corruption, having to do with appending data to a string BAT was fixed.

Bug Fixes

4017: server crashes when executing particular loopback query in embedded python
6207: identifier ambiguous when grouping and selecting the same column twice
6239: Incorrect profiling
6261: New handling of delta tables hurts badly reusage of bats
6287: should the CORR function return some numeric type that allows fractions?
6321: Two-column aggregation on join result extremely slow.
6335: Sqlitelogictest crash in complex SQL query
6343: MERGE TABLE issue: unable to find
6348: Interference of procedure/table name
6350: Carriage return and form feed in TEXT fields are returned as ‘r’ and ‘f’ in jdbcclient and ResultSets
6352: Scope resolution problem (sqlsmith)
6353: implicit NULL value not propagated in distributed/remote query
6374: Wrong answer from merge table after content changes
6379: Table UDF: SEGV raised when invoking a non existing function
6380: unable to create new databases from clean installation
6381: Parser misses error messages in conditional
6382: Can’t set JSON fields via PreparedStatement
6384: crash when setting a wrong listenaddr
6385: AGGREGATE UDFs with more than 2 parameters incorrectly processed
6386: Unexpected error from server for query with long floats
6387: Performance degradation on multi column sort
6388: JDBC Connection via user voc produces errors when fetching certain meta data information
6392: SELECT EXISTS (empty table) returns ’true’
6395: BAT leak of scalar result sets
6397: Isolation of generating functions not correct
6398: Null Matches in outer join are not supported
6399: UDF crashes when subquery and scalar values are passed as pameters
6400: getCharacterStream() currently not supported
6404: COPY INTO crashes if table has primary key or foreign key constraint
6405: Compilation error if DISABLE_PARENT_HASH not defined
6409: sqllogictest crash on aggregation query with NOT IN clause in HAVING clause
6410: Sqlitelogictest crash on aggregation query with IN clause
6411: Sqlitelogictest crash in aggregation query
6412: Overflow in sys.epoch
6413: Wrong answer for TPC-H Q17
6414: Using RPAD returns: could not allocate space
6416: Sqlitelogictest crash in aggregation query with a NOT IN clause
6417: Segfault encountered (sqlsmith)
6418: Segfault in renaming (sqlsmith)
6419: segfault in rel_optimizer (sqlsmith)
6420: Assertion error in mergetable task (sqlsmith)
6422: Another assertion error in rel_or (sqlsmith)
6423: Dereference null pointer (sqlsmith)
6424: Assertion error in rel_rename_expr (sqlsmith)
6425: Assertion error in exp_bin (sqlsmith)
6426: Assertion error in rel_find_exp_ (sqlsmith)
6427: Assertion error in eq_typeswitchloop (sqlsmith)
6428: Sqlitelogictest crash in aggregation query
6430: Assertion raised in another eq_typeswitch error (sqlsmith)
6431: Sqlitelogictest crash in aggregation query with a long having clause
6432: Assertion error in exp_bin (sqlsmith)
6433: Sqlitelogictest crash in complex SELECT query with IN operator
6435: Sqlitelogictest crash in simple select query
6437: System schemas “profiler” and “json” shouldn’t be allowed to be dropped.
6438: Implement functionality to enforce the restrict option in: DROP SCHEMA xyz RESTRICT;
6440: Faulty plan generated. Query returns more rows than expected or existing in the view sys.tables.

Important: we found a bug in the Jul2017 and Jul2017-SP1 releases (only) that can corrupt your database. We recommend to not upgrade to these two releases, but to skip directly to the Jul2017-SP2 release. See this email message.

Jul 2017-SP1 bugfix release (11.27.5)

Debian and Ubuntu uses may want to reimport the MonetDB GPG public key as per the instructions on the download page. See bug 6383 for more information.

Build Environment

The Debian and Ubuntu installers have been fixed: there was a file missing in the Jul2017 release.
Added a new RPM called MonetDB-selinux which provides the SELinux policy required to run MonetDB under systemd, especially on Fedora 26.
The Windows installers (*.msi files) are now created using the WiX Toolset.
The Windows binaries are now built using Visual Studio 2015. Because of this, you may need to install the Visual C++ Redistributable for Visual Studio 2015 before being able to run MonetDB.

Merovingian

monetdbd was leaking open file descriptors to the mserver5 process it started. This has been fixed.

MonetDB Common

Many functions in GDK are now annotated with the GCC attribute warn_unused_result meaning that the compiler will issue a warning if the result of the function (usually an indication of an error) is not used.

Bug Fixes

6325: Merge table unusable in other connections
6328: Transactional/multi-connection issues with merge tables
6336: VALUES multiple inserts error
6339: Mserver5 crashes on nested SELECT
6340: sample operator takes effect after the execution of the query, expected before
6341: MERGE TABLE issue: Cannot register
6342: MERGE TABLE issue: hang
6344: Spurious errors and assertions (SQLsmith)
6375: MAL profiler truncates JSON objects larger than 8192 characters

Jul 2017 feature release (11.27.1)

On Fedora 26 there is a known issue with this release. When you use MonetDB under systemd (if you have enabled the system service with systemctl enable monetdbd.service), and you have SELinux (Security-Enhanced Linux) enabled, the database server processes will not start when you try to connect.

MonetDB5 Server

The “sub” prefix of many functions, both at the MAL and the C level, has been removed.
Changed the interfaces of the AUTH* functions: pass values, not pointers to values.
Removed calc.setoid().
group.subgroup is now called group.group if it is not refining a group. Both group.group and group.subgroup now also have variants with a candidate list.
The allocation schemes for MAL blocks and Variables has been turned into block-based. This reduces the number of malloc()/free() calls.
Added a new server-side protocol implementation. The new protocol is backwards compatible with the old protocol. Clients can choose whether they want to use the old or the new protocol during the initial handshake with the server. The new protocol is a binary column-based protocol that is significantly faster than the old protocol when transferring large result sets. In addition, the new protocol supports compression using Snappy or LZ4.
Moved the sphinx extension module to its own repository. See https://dev.monetdb.org/hg/MonetDB-sphinx/.
Removed GSL module: it’s now a separate (extension) package. See https://dev.monetdb.org/hg/MonetDB-gsl/.
The PCRE library is now optional for systems that support POSIX regular expressions.
Added 5 new sys schema tables: function_languages, function_types, key_types, index_types and privilege_codes. They are pre-loaded with static content and contain descriptive names for the various integer type and code values. See also sql/scripts/51_sys_schema_extension.sql

Merovingian

Added handling of a dbextra property per database at the daemon level. The user can set the dbextra property for a database using the command: $ monetdb set dbextra= and the daemon will make sure to start the new server using the correct –dbextra parameter.

Client Package

The mclient and msqldump programs lost compatibility with old mserver5 versions (pre 2014) which didn’t have a “system” column in the sys.schemas table.
The mclient and msqldump programs lost compatibility with ancient mserver5 versions (pre 2011) which didn’t have the sys.systemfunctions table.
Removed the “array” and “quick” functions from the mapi library. To be precise, the removed functions are: mapi_execute_array, mapi_fetch_field_array, mapi_prepare_array, mapi_query_array, mapi_quick_query, mapi_quick_query_array, and mapi_quick_response.
Added a more elaborate \help command for SQL expressions.

MonetDB Common

Improved error checking in the logger code (dealing with the write-ahead log); changed return types a several functions from int to gdk_return (i.e., they now return GDK_SUCCEED or GDK_FAIL). The logger no longer calls GDKfatal on error. Instead the caller is responsible for dealing with errors.
BATsort may now create an order index as a by product.
Quantile calculations now use the order index if available (and use BATsort otherwise, producing an order index).
Quantiles calculate a position in the sorted column. If this position is not an integer, we now choose the nearest position, favoring the lower if the distance to the two adjacent positions is equal (round down to nearest integer).
Removed function BATprintf. Use BATprint or BATprintcolumns instead.
Removed BATsave from the list of exported functions.
Replaced BBPincref/BBPdecref with BBPfix/BBPunfix for physical reference count and BBPretain/BBPrelease for logical reference count maintenance.
Removed automatic conversion of 32-bit OIDs to 64 bits on 64-bit architectures.
Removed functions OIDbase() and OIDnew().
Removed talign field from BAT descriptor.
BATappend now takes an optional (NULL if not used) candidate list for the to-be-appended BAT.
New function BATkeyed(BAT *b) that determines (possibly using a hash table) whether all values in b are distinct.

SQL

Made the operator precedence of % equal to those of * and /. All three are evaluated from left to right.
Removed table sys.connections. It was a remnant of an experimental change that had already been removed in 2012.
Protect against runaway profiler events If you hit a barrier block during profiling, the JSON event log may quickly become unwieldy. Event production is protected using a high water mark, which ensures that never within the single execution of MAL block the instruction causes excessive event records.

Bug Fixes

3465: Request: add support for CREATE VIEW with ORDER BY clause
3545: monetdb commands don’t work with -h -P -p options (locally and remotely)
3996: select * from sys.connections always returns 0 rows. Expected to see at least one row for the active connection.
6187: Nested WITH queries not supported
6225: Order of evaluation of the modulo operator
6289: Crashes and hangs with remote tables
6292: Runaway SQL optimizer in too many nested operators
6310: Name resolution error (sqlsmith)
6312: Object not found in LIMIT clause (sqlsmith)
6313: Null type resolution in disjunction fails (sqlsmith)
6319: Server crash on LATERAL (sqlsmith)
6322: Crash on disjunction with LIMIT (sqlsmith)
6323: Deadlock calling sys.bbp()
6324: Sqlitelogictest crash in a IN query (8th)
6327: The daemon does not respect the actual name of the mserver5 executable
6330: Sqlitelogictest crash on a complex SELECT query
6331: sys.statistics column “nils” always contains 0. Expected a positive value for columns that have one or more nils/NULLs
6332: Sqlitelogictest crash related to an undefined MAL function