- announce-list - monetdb.org

Announcement: New Aug2018 Feature release of MonetDB suite
by Sjoerd Mullender 31 Aug '18

31 Aug '18

The MonetDB team at CWI/MonetDB BV is pleased to announce the Aug2018 feature release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <https://www.monetdb.org/>. For details on this release, please see the release notes at <https://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <https://dev.monetdb.org/downloads/>. Aug 2018 feature release (11.31.7) MonetDB5 Server * The lsst module was moved to a separate repository (https://dev.monetdb.org/hg/MonetDB-lsst/) Build Environment * Build the MonetDB-cfitsio RPM and libmonetdb5-server-cfitsio Debian/Ubuntu package. * On Windows, the separate MonetDB5-Geom installer has been incorporated into the main MonetDB5-SQL installer and is therefore no longer available as a separate download. Merovingian * Added a "logrotate" configuration file. See /etc/logrotate.d/monetdbd. * Changed the monetdb profilerstart command to be more robust. If the server or stethoscope crashed before, the pid file is still there, so the next time we try to start stethoscope, it will fail. Now the profilerstart command will check if a stethoscope process with the recorded pid is running. If not, we start stethoscope, assuming that something went wrong before. * Changed the monetdb stop command to try to stop stethoscope before stoping the server. The error conditions that can arrise from attempting to stop stethoscope are: + The database is not running. + The profilerlogpath is not set. + The profiler.pid file does not exist or is inaccessible. + The contents of the profiler.pid are not valid. + Shutdown of stethoscope did not succeed. + Removing the profiler.pid file failed. In all the cases, the attempt to stop the server can continue normally, so we actually ignore any errors that rise from the attempt to stop stethoscope. Client Package * ODBC: Implemented SQL_ATTR_QUERY_TIMEOUT parameter in SQLSetStmtAttr. * ODBC SQLGetInfo now returns a positive numeric value for InfoTypes: SQL_MAX_COLUMN_NAME_LEN, SQL_MAX_DRIVER_CONNECTIONS, SQL_MAX_IDENTIFIER_LEN, SQL_MAX_PROCEDURE_NAME_LEN, SQL_MAX_SCHEMA_NAME_LEN, SQL_MAX_TABLE_NAME_LEN and SQL_MAX_USER_NAME_LEN. * Added a '-f rowcount' option in mclient to repress printing the actual data of a resultset, but only print the number of returned tuples Stream Library * Added support for lz4 compressed files in the stream library MonetDB Common * Hash indexes are now persistent across server restarts. * The macros bunfastapp and tfastins and variants no longer set the dirty flag of the heap they write to. This now needs to be done separately (and preferably outside of the inner loop). * Removed batDirty flag from BAT record. Its function is completely superseded by batDirtydesc and the dirty flags on the various heaps. * Removed "masksize" argument of function BAThash. * A whole bunch of functions that took an int argument that was used as a Boolean (true/false) value now take a value of type bool. The functions BATkeyed, BATordered and BATordered_rev now return a bool instead of an int. * Removed the tdense property: it's function is completely replaced by whether or not tseqbase is equal to oid_nil. Testing Environment * Removed helper programs Mtimeout and MkillUsers: they have long been superseded by timeout handling by Mtest.py itself. SQL Frontend * Removed deprecated table producing system functions: sys.dependencies_columns_on_functions() sys.dependencies_columns_on_indexes() sys.dependencies_columns_on_keys() sys.dependencies_columns_on_triggers() sys.dependencies_columns_on_views() sys.dependencies_functions_on_functions() sys.dependencies_functions_on_triggers() sys.dependencies_keys_on_foreignkeys() sys.dependencies_owners_on_schemas() sys.dependencies_schemas_on_users() sys.dependencies_tables_on_foreignkeys() sys.dependencies_tables_on_functions() sys.dependencies_tables_on_indexes() sys.dependencies_tables_on_triggers() sys.dependencies_tables_on_views() sys.dependencies_views_on_functions() sys.dependencies_views_on_triggers() They are replaced by new system dependency_* views: sys.dependency_args_on_types sys.dependency_columns_on_functions sys.dependency_columns_on_indexes sys.dependency_columns_on_keys sys.dependency_columns_on_procedures sys.dependency_columns_on_triggers sys.dependency_columns_on_types sys.dependency_columns_on_views sys.dependency_functions_on_functions sys.dependency_functions_on_procedures sys.dependency_functions_on_triggers sys.dependency_functions_on_types sys.dependency_functions_on_views sys.dependency_keys_on_foreignkeys sys.dependency_owners_on_schemas sys.dependency_schemas_on_users sys.dependency_tables_on_foreignkeys sys.dependency_tables_on_functions sys.dependency_tables_on_indexes sys.dependency_tables_on_procedures sys.dependency_tables_on_triggers sys.dependency_tables_on_views sys.dependency_views_on_functions sys.dependency_views_on_procedures sys.dependency_views_on_views * Implemented group_concat(X,Y) aggregate function which also concatenates a column of strings X, but using a supplied string Y as the separator. This function is also a SQL extension. * Implemented group_concat(X) aggregate function which concatenates a column of strings using a comma as a separator. This function is not featured in the SQL standard. Bug Fixes * 4020: Importing timestamp with zone from copy into * 6506: Improper performance counters * 6556: Sqlitelogictest division by zero on COALESCE call * 6564: Changes to the Remote Table definition * 6575: Sqlitelogictest crash on groupby query with coalesce call * 6579: Sqlitelogic test infinite loop while compiling SQL query * 6586: Sqlitelogictest crash on complex aggregation query * 6593: Poor performance with like operator and escape clause * 6596: Multicolumn aggregation very slow after ANALYZE when persistent hashes are enabled * 6605: Sqlitelogictest set queries with wrong results * 6606: Misleading parameter name in generate_series function * 6610: Sqlitelogictest algebra.rangejoin undefined * 6611: Cannot compile with GCC 8.1 and --enable-debug=no * 6612: Implement BLOB handling in python UDFs * 6618: dependency column on sequence violated by DROP SEQUENCE * 6621: SELECT FROM REMOTE TABLE WHERE <> returns wrong results * 6624: "Cannot use non GROUP BY column in query results without an aggregate function" when using aggregate function in both HAVING and ORDER BY clauses. * 6625: OR in subselect causes the server to crash with segmentation fault * 6627: stddev_pop inconsistent behaviour * 6628: User cannot insert into own local temporary table * 6629: CREATE TABLE IF NOT EXISTS returns 42000! * 6630: Sqlitelogictest cast NULL to integer failing * 6632: Dataflow causes crash when THRnew fails * 6633: ILIKE clauses don't work on certain characters * 6635: monetdbd exits due to "Too many open files" error * 6637: Within a transaction, \d after an error causes mclient to exit * 6638: (sequences of) mkey.bulk_rotate_xor_hash() can generate NIL from non-NIL making multi-col joins return wrong results * 6639: COMMENT ON TABLE abc IS NULL invalidly sets the remark column to null where remark column is defined as NOT NULLable

1 0

Announcement: New Mar2018-SP1 Bugfix release of MonetDB suite
by Panagiotis Koutsourakis 20 Jun '18

20 Jun '18

The MonetDB team at CWI/MonetDB BV is pleased to announce the Mar2018-SP1 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <https://www.monetdb.org/>. For details on this release, please see the release notes at <https://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <https://dev.monetdb.org/downloads/>. Mar 2018-SP1 bugfix release (11.29.7) Build Environment * On Linux and Unix, try not to link to libraries multiple times: that only causes the code (and worse, the variables in those libraries) to be included multiple times. On Windows, we do need to link to libraries multiple times (once for each DLL we create if the DLL references anything from the library) but that doesn't cause the linked library to be loaded multiple times. SQL Frontend * Corrected the definition of view: sys.ids. Bug Fixes * 6562: Sqlitelogictest crash on group by query with not in operator * 6565: Sqlitelogictest crash on complex select query with coalesce call * 6566: Sqlitelogictest unavailable calc.- MAL operations * 6568: Sqlitelogictest crash on complex case query * 6569: Sqlitelogictest select query with not between cause with wrong results * 6570: Sqlitelogictest select coalesce undefined calc * 6572: ordered index Error in optimizer garbageCollector * 6573: Sqlitelogictest crash on complex select query with case statement * 6574: server crashed could not find ordered index while creating a table * 6576: Sqlitelogictest aritmetic expressions with negative numbers handling * 6577: creating temp table kills performance of the original query * 6578: One two-tuple insert gives different results than two single inserts * 6581: Join condition errors. * 6583: Fixed size string concatenation with integer results in fixed size string of size 0 * 6584: SELECT FROM REMOTE TABLE WHERE IS NOT NULL produces wrong result * 6585: Nested Merge tables cause an infinite loop in rel_optimizer * 6587: Sqlitelogictest crash on complex case statement * 6589: Sqlitelogictest crash on complex on complex expression * 6594: Sqlitelogictest crash on complex case statement * 6595: Remote decimal division triggers assertion / returns wrong answer * 6598: Python 3.4 not supported (due to usage of Py_DecodeLocale) * 6600: Sqlitelogictest queries fail to execute * 6601: "where is null" clause on remote table causes problem with next query * 6602: Sqlitelogictest wrong results in IN query * 6603: Sqlitelogictest: Aggregation query with distinct clause produces duplicated rows

1 0

Announcement: New Mar2018 Feature release of MonetDB suite
by Sjoerd Mullender 29 Mar '18

29 Mar '18

The MonetDB team at CWI/MonetDB BV is pleased to announce the Mar2018 feature release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <https://www.monetdb.org/>. For details on this release, please see the release notes at <https://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <https://dev.monetdb.org/downloads/>. Some notable changes: - Code base requires a C99 compiler. - The internal representation of REAL and DOUBLE NULL values has changed. Before they were represented as the smallest C float/double finite value (-FLT_MAX and -DBL_MAX). Now they are represented by a NaN (not-a-number) value. This requires a database upgrade which is done automatically at startup. Be advised that it is recommended to make a backup before upgrading. - Various extensions to SQL have been implemented. For more details, see below. Mar 2018 feature release (11.29.3) MonetDB5 Server * Implemented function pcre.replace_first which is like pcre.replace, except it only replaces the first match. * The EXPLAIN command now shows all the MAL type resolutions, because in general users may not be aware of their signatures. It also simplifies programs to analyze such plans. * Implemented versions of group.(sub)group(done) that don't return a histogram. * Removed MAL functions streams.socketRead, streams.socketReadBytes, streams.socketWrite, and streams.socketWriteBytes. * Removed MAL functions streams.openRead(s:streams):streams and streams.openWrite(s:streams):streams. * Lots of changes to streamline the internal error handling. In principle, all errors should now include a SQLSTATE error code (see the SQL standard). Build Environment * Added the .pdb files needed for debug symbols to the Windows installer for MonetDB/SQL. . * mclient's execution time profiling options and output format have been changed. Instead of implicitly via the "--interactive"/"-i" option (or when using an interactive mclient console), execution time profiling is now controlled via an explicit "--timer=timermode"/"-t timermode" command-line option, or a "\t timermode" command in the interactive mclient console. The default (also in the interactive mclient console) is now timermode "none", i.e., no timing information is given. Timermode "clock" activates client-side wall-clock timing ("clk") in "human-friendly" format much like the interactive mode did before. Timermode "performance" also provides detailed server-side timings: "sql" is the time to parse the SQL query, optimize the logical relational plan and create the initial physical (MAL) plan; "opt" is the time to optimize the physical (MAL) plan; "run" is the time to execute the physical (MAL) plan. With timermode "performance" all server-side timings and the client-side wall-clock time are given in milliseconds (ms). Note that the client-measured wall-clock time "clk" is reported per query only when options "--interactive" or "--echo" are used, because only then does mclient send individual lines (statements) of the SQL script to the server. Otherwise, mclient sends the SQL script in large(r) batch(es) to the server, and, thus, only the total wall-clock time per batch is measured and reported. The server-measured detailed performance timings "sql", "opt", "run" are always measured and reported per query. Also, all timing information is now given on a separate line and sent to stderr rather than stdout. * Some types and constants were moved from configure (and hence monetdb_config.h) to gdk.h. In particular, the types "lng" and "ulng" have been moved and can therefore no longer be used by code that doesn't (ultimately) include gdk.h. Just use int64_t instead. A bunch of format defines have been removed: SZFMT, SSZFMT, PTRFMT, PDFMT. Just use the C standard codes for those (%zu, %zd, %p, %td). The define for printing a lng (LLFMT) was also moved. Use PRId64 for printing int64_t in code not using gdk.h. Removed all references to __int64 and long long (use int64_t instead). Merovingian * Add daemon commands for starting/stopping collection of profiler (stethoscope) logs. Client Package * ODBC: The driver function SQLProcedureColumns was implemented. * ODBC: Changed table types as used by SQLTables from "LOCAL TEMPORARY" and "GLOBAL TEMPORARY" to "LOCAL TEMPORARY TABLE" and "GLOBAL TEMPORARY TABLE" respectively. * ODBC: Initial support for the HUGEINT type in SQL was added. Any value with type HUGEINT can be retrieved in a C variable with type SQL_C_CHAR or SQL_C_WCHAR. A value of type HUGEINT can be retrieved in other C types as long as they fit, the largest C type supported being a 64 bit integer (equivalent to BIGINT). * The functions in the mapi library that require 64 bit integers now use the standard type int64_t instead of the non-standard mapi_int64. This requires a compilation environment that has the stdint.h include file (standardized in C99). Compilation of the library also requires the inttypes.h include file (also standardized in C99). * Add a new pretty printing option to stethoscope Running stethoscope with the flag -j will produce not pretty printed output (one json object per line). Running with the -y flag will produce pretty printed output. Running with neither will produce the legacy, line oriented format mapilib * The three mapi_explain* functions and mapi_trace don't return any useful information, so they now return void. * The functions mapi_error_str and mapi_result_error now return const char * instead of plain char * to indicate that the returned data belongs to the library and should not be changed or freed by the application. * New function const char *mapi_result_errorcode(MapiHdl) which returns the SQLSTATE code if available when an error has occurred. stream * The interface of mnstr_fgetpos and mnstr_fsetpos was changed to look more like the standard C functions fsetpos and fgetpos: they both have a second argument "pointer to fpos_t". * Removed function wbstream. * Removed functions udp_rastream and udp_wastream. * Removed functions socket_rstream and socket_wstream. * Removed functions append_wstream and append_wastream. * Removed functions mnstr_rstream and mnstr_wstream. utils * The function mcrypt_getHashAlgorithms now returns a static, constant string, so the result should not be modified or freed. MonetDB Common * Changed return type of function void_replace_bat from BUN to gdk_return: it now only returns whether the operation succeeded or not. * Changed the return type of BATroles from void to gdk_return: it can fail due to malloc failure. * Removed functions ALIGNsetH, ALIGNsetT, and CREATEview_ (mind the underscore). The first can easily be replace by using BAThseqbase (that's all it did), the second was only used once, and the third can be replace by VIEWcreate. * Removed unused functions BATmemsize and BATvmsize. * Removed the tnodense property: it was maintained but never actually used, not even stored. * The NIL representation of the internal flt and dbl types was changed from the smallest representable finite value to NaN (not-a-number). * Changed the interface of ATOMformat and VALformat: they now return a pointer to the allocated string. * The length "method" for atoms now returns a size_t, the "len" field of a ValRecord is now a size_t, the "size" field of the atomDesc structure is now unsigned short. * Removed the "align" field from the ATOM descriptor (atomDesc) structure. * The atomtostr and atomfromstr "methods" for atoms now return ssize_t and require a pointer to size_t for the size of the buffer. * The atom tostr and fromstr "methods" now always return -1 on error. A return value greater than 0 is normal, a return value of 0 is not normal, but technically not an error. Testing Environment * Added a --data_path option to Mtest.py that defines an external data repository. See the commit message of c484932c7fd8 for more info. SQL * Extended support to use CREATE ORDERED INDEX on columns of type: char, varchar, clob, blob, url, json, inet and uuid. * Added new system view: sys.ids which contains all database objects ids which can be used in sys.dependencies table. * Added new system view: sys.dependencies_vw which shows all data of sys.dependencies including names on objects, object types and dependency types. * Added 25 new system views for finding out dependencies between database objects. These new dependency views improve, extend and replace the 17 sys.dependencies_X_on_Y() functions as previously defined in 21_dependency_functions.sql. Those sys.dependencies_X_on_Y() functions are now marked as deprecated. * Added new system view: sys.roles which contains all defined roles. * Added new system view: sys.var_values which shows the values for system variables. * Added support for COMMENT ON statements using SQL syntax: COMMENT ON { SCHEMA | TABLE | VIEW | COLUMN | INDEX | SEQUENCE | FUNCTION | PROCEDURE | AGGREGATE | FILTER FUNCTION | LOADER } qname IS { 'my description text' | NULL | '' } ; For COLUMN the qname can be "table_name"."column_name" or fully qualified as in: "schema_name"."table_name"."column_name". For FUNCTION, PROCEDURE, AGGREGATE, FILTER FUNCTION and LOADER the qname may need to include the signature (argument types) to be able to differentiate between multiple overloaded functions which have the same name and schema. By specifying IS NULL or IS '' you remove the comment for the database object. If a database object is dropped, the associated comment is also removed. Note: it is not allowed or possible to add comments for temporary tables or objects in schema "tmp". The sql catalog has been extended with system table: sys.comments. The keyword 'COMMENT' has now become a reserved keyword. * Removed system function sys.environment(). It was a duplicate of system function sys.env(). * Implemented behavior for DROP SCHEMA my_schema RESTRICT command. Previously the RESTRICT keyword was accepted but not obeyed. It would always do a CASCADE operation. Also the default behavior of DROP SCHEMA my_schema command is now changed into RESTRICT behavior (was CASCADE). * The internal NULL representation of the REAL (FLOAT) and DOUBLE types was changed from the smallest representable finite value to NaN (not-a-number). * A column default value can be used in a UPDATE statement: UPDATE tname SET cname = DEFAULT, and INSERT statements: INSERT INTO tname VALUES (..., DEFAULT, ...) * Added support for TRUNCATE statements conforming to the SQL:2008 standard: TRUNCATE [ TABLE ] qname [ CONTINUE IDENTITY | RESTART IDENTITY ] [ RESTRICT | CASCADE ] In a TRUNCATE statement a 'CONTINUE IDENTITY' or 'RESTART IDENTITY' clause can be passed to restart or not, being the former the default one. The 'CASCADE' option instructs to truncate referencing table(s) also if the referencing table(s) have foreign key references to this table. The default behavior is 'RESTRICT'. Note: it is possible to use TRUNCATE statements in a transaction and thus to rollback the effects of a truncate. The keywords 'TRUNCATE' and 'CONTINUE' have now become reserved keywords. * Added possibility to GRANT or REVOKE a TRUNCATE privilege to a user or role. * Added possibility to define a TRIGGER on a TRUNCATE event. * Added possibility to specify 'OR REPLACE' in following CREATE commands: CREATE [ OR REPLACE ] VIEW qname ... CREATE [ OR REPLACE ] TRIGGER qname ... * Added possibility to specify 'IF EXIST' in following DROP commands: DROP AGGREGATE [ IF EXISTS ] qname ... DROP FUNCTION [ IF EXISTS ] qname ... DROP FILTER FUNCTION [ IF EXISTS ] qname ... DROP LOADER [ IF EXISTS ] qname ... DROP PROCEDURE [ IF EXISTS ] qname ... DROP TRIGGER [ IF EXISTS ] qname ... * Lots of changes to streamline the internal error handling. In principle, all errors should now include a SQLSTATE error code (see the SQL standard). * Added support for extracting the quarter (number between 1 and 4) of a date or a timestamp or a timestamp with timezone in SQL: EXTRACT ( QUARTER FROM my_date_expr ). Added support for extracting the week (number between 1 and 53) of a date or a timestamp or a timestamp with timezone in SQL: EXTRACT ( WEEK FROM my_date_expr ). Added support for scalar functions: quarter(date_expr), quarter(timestamp_expr) and quarter(timestamptz_expr). Bug Fixes * 3574: Add support for: create OR REPLACE view ... * 3824: Created table not visible from ODBC * 3831: Extend date part extraction and date formating functions to support more formats like "quarter", "iso year/week" * 6244: Add support for: TRUNCATE TABLE * 6249: DEFAULT in row-values missing (sqlsmith) * 6346: BATsort returns GDK_SUCCEED when **sorted bat is null * 6438: Implement functionality to enforce the restrict option in: DROP SCHEMA xyz RESTRICT; * 6507: Column Header coming with trailing spaces in compiled version of MonetDB 11.27.9/11 * 6513: Sqlitelogictest: Wrong MAL plan generation for column product * 6526: Crash using aggregate function inside a case statement in having clause * 6529: Sqlitelogictest crash in select query with IN operator and cast * 6530: Sqlitelogictest: select query with NOT IN giving wrong results * 6532: copy into ignore null as directive if first column doesn't come from file * 6534: [Mar2018]: mclient -f tab / --format=tab complains about "unsupported formatter" * 6535: [Mar2018]: mclient -t / --timer does not work as documented * 6536: [Mar2018]: timing output of mclient -t / --timer= should go to stderr rather than stdout * 6537: [Mar2018]: mclient's default timing mode should (again) be "none" rather than "clock" * 6541: [Mar2018]: mclient reports incorrect wall-clock time * 6542: assertion failure when querying: select count(*) from sys.commented_function_signatures; * 6543: Mar2018: truncate on SQL system tables should NOT be allowed * 6545: Sqlitelogictest crash in IN query * 6546: Sqlitelogictest crash in IN query with division * 6547: OS-dependent behaviour for ilike * 6548: Select from remote table leaves session open * 6549: Add log messages when listen fails * 6550: Sqlitelogictest crash on complex CASE statement * 6551: Sqlitelogictest wrong NULL value cast * 6552: Sqlitelogictest crash on complex case statement * 6553: Sqlitelogictest crash on aggregation with having statement * 6554: Sqlitelogictest crash on complex case statement * 6555: Sqlitelogictest wrong result set generated from complex case statement * 6556: Sqlitelogictest division by zero on COALESCE call * 6557: Sqlitelogictest crash on aggregation query with not in * 6559: rows in sys.statistics are not removed when a temporary table is dropped. * 6560: Sqlitelogictest crash on group by query with having in * 6561: Sqlitelogictest crash on group by query with having not in clause

1 0

Announcement: New Jul2017-SP4 Bugfix release of MonetDB suite
by Sjoerd Mullender 12 Feb '18

12 Feb '18

The MonetDB team at CWI/MonetDB BV is pleased to announce the Jul2017-SP4 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <http://www.monetdb.org/>. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Jul 2017-SP4 bugfix release (11.27.13) Build Environment * Added the .lib and .h files needed for building extensions to the Windows installer. Bug Fixes * 3470: Support setClob without length restrictions * 6468: JDBC 2.27 fails with year < 1000 * 6482: Query failures on order by on union * 6483: Monetdb crashes, on query * 6487: UNION of NULLs with several tables fails * 6488: Semijoin returns duplicate values from a column with unique values * 6489: Sqlitelogictest - Wrong result set of complex conditional query * 6490: Sqlitelogictest - Select query with an IN clause parse error * 6491: SELECT IN returns NULL instead of false when not found * 6492: Persistent hashes stored and then ignored. Storage info not in sync with actual indices. * 6493: Sqlitelogictest - Aggregation query on empty table with DISTINCT clause * 6494: Sqlitelogictest - Algebra operators priority in select query * 6495: Sqlitelogictest - Omitting AS in a result set column alias name * 6496: Sqlitelogictest - Select interval comparisons between floating-points and NULL * 6497: Sqlitelogictest - Select not between query producing wrong results * 6498: large virtual memory spike on BLOB column COUNT * 6499: Crash when trying to replace a function defined in sys from a different schema * 6502: Query with multiple limit clauses does not return anything * 6508: Segmentation fault in mserver5 on Python2 UDF with TIMESTAMP column input that has NULL values (conversion.c:438, PyNullMask_FromBAT) * 6510: Sqlitelogictest: Wrong output in aggregation query * 6512: Monetdb crashes on query with limit after sort with case * 6514: Sqlitelogictest: Range query between NULL values not possible * 6515: Insert null second interval value results in 0 * 6516: Sqlitelogictest unknown bat append operation * 6517: Sqlitelogictest overflow in conversion during MAL plan execution * 6518: Sqlitelogictest: count aggregation with not in operator * 6519: Sqlitelogictest: algebra join between lng and int BATs undefined * 6520: UPDATE with correlated subquery causes assertion (or segfault) * 6522: Sqlitelogictest: IN operator return a single column * 6523: Sqlitelogictest: Case statement subquery missing * 6524: Sqlitelogictest: Crash in aggregation query with IN operator * 6527: Crash using order by alias in subquery

1 0

Announcement: New Jul2017-SP3 Bugfix release of MonetDB suite
by Sjoerd Mullender 14 Dec '17

14 Dec '17

The MonetDB team at CWI/MonetDB BV is pleased to announce the Jul2017-SP3 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <http://www.monetdb.org/>. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Jul 2017-SP3 bugfix release (11.27.11) MonetDB Common * Reimplemented summing of a column of floating point (flt and dbl) values. The old code could give wildly inaccurate results when adding up lots and lots of values due to lack of precision. Try SELECT sum(c) FROM t; where t is 100,000,000 rows, c is of type REAL and all values are equal to 1.1. (The old code returned 33554432 instead of 1.1e8.) Bug Fixes * 3898: Deadlock on insertion * 6429: ROUND produces wrong data type * 6436: Query sequence with 2x ifthenelse() and next nullif() causes mserver5 Segmentation fault * 6437: System schemas "profiler" and "json" shouldn't be allowed to be dropped. * 6439: Invalid references to sys.columns.id from sys.statistics.column_id * 6442: SEGFAULT with COPY INTO BEST EFFORT and skipping input columns * 6443: complex(?) query forgets(?) column name * 6444: Using 'with' keyword with table returning function crashes monetdb * 6445: Sqlitelogictest crash in MySQL query * 6446: sql_parser.y bug? * 6448: 'insert into' with multiple rows containing subqueries crashes * 6449: Assertion error in rel_dce_refs (sqlsmith) * 6450: Assertion error in exp_bin (sqlsmith) * 6451: Assertion error in sql_ref_dec (sqlsmith) * 6453: Assertion error in rel_rename_exps (sqlsmith) * 6454: SQL lexical error * 6455: Assertion error in rel_apply_rewrite (sqlsmith) * 6456: NULL becomes 0 in outer join * 6459: Assertion error in exp_bin (sqlsmith) * 6462: large virtual memory spike on BLOB column select * 6465: appending to variables sized atom bats other than str bats with force flag may result in corrupted heap * 6467: date_to_str formatter is wrong * 6470: mitosis gets in the way of simple select * 6471: calls to sys.generate_series should auto-convert arguments * 6472: Assertion failure in rel_rename (Sqlsmith) * 6477: assertion eror rel_push_project_up (sqlsmith) * 6478: Crash with nested order by/ limit offset * 6479: Mserver receives an assertion error on a procedure call * 6480: Segfault in mvc_find_subexp (sqlsmith)

1 0

Announcement: New Jul2017-SP2 Bugfix release of MonetDB suite
by Sjoerd Mullender 26 Oct '17

26 Oct '17

The MonetDB team at CWI/MonetDB BV is pleased to announce the Jul2017-SP2 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <http://www.monetdb.org/>. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Jul 2017-SP2 bugfix release (11.27.9) MonetDB Common * A serious bug, possibly resulting in database corruption, having to do with appending data to a string BAT was fixed. Bug Fixes * 4017: server crashes when executing particular loopback query in embedded python * 6207: identifier ambiguous when grouping and selecting the same column twice * 6239: Incorrect profiling * 6261: New handling of delta tables hurts badly reusage of bats * 6287: should the CORR function return some numeric type that allows fractions? * 6321: Two-column aggregation on join result extremely slow. * 6335: Sqlitelogictest crash in complex SQL query * 6343: MERGE TABLE issue: unable to find * 6348: Interference of procedure/table name * 6350: Carriage return and form feed in TEXT fields are returned as 'r' and 'f' in jdbcclient and ResultSets * 6352: Scope resolution problem (sqlsmith) * 6353: implicit NULL value not propagated in distributed/remote query * 6374: Wrong answer from merge table after content changes * 6379: Table UDF: SEGV raised when invoking a non existing function * 6380: unable to create new databases from clean installation * 6381: Parser misses error messages in conditional * 6382: Can't set JSON fields via PreparedStatement * 6384: crash when setting a wrong listenaddr * 6385: AGGREGATE UDFs with more than 2 parameters incorrectly processed * 6386: Unexpected error from server for query with long floats * 6387: Performance degradation on multi column sort * 6388: JDBC Connection via user voc produces errors when fetching certain meta data information * 6392: SELECT EXISTS (empty table) returns 'true' * 6395: BAT leak of scalar result sets * 6397: Isolation of generating functions not correct * 6398: Null Matches in outer join are not supported * 6399: UDF crashes when subquery and scalar values are passed as pameters * 6400: getCharacterStream() currently not supported * 6404: COPY INTO crashes if table has primary key or foreign key constraint * 6405: Compilation error if DISABLE_PARENT_HASH not defined * 6409: sqllogictest crash on aggregation query with NOT IN clause in HAVING clause * 6410: Sqlitelogictest crash on aggregation query with IN clause * 6411: Sqlitelogictest crash in aggregation query * 6412: Overflow in sys.epoch * 6413: Wrong answer for TPC-H Q17 * 6414: Using RPAD returns: could not allocate space * 6416: Sqlitelogictest crash in aggregation query with a NOT IN clause * 6417: Segfault encountered (sqlsmith) * 6418: Segfault in renaming (sqlsmith) * 6419: segfault in rel_optimizer (sqlsmith) * 6420: Assertion error in mergetable task (sqlsmith) * 6422: Another assertion error in rel_or (sqlsmith) * 6423: Dereference null pointer (sqlsmith) * 6424: Assertion error in rel_rename_expr (sqlsmith) * 6425: Assertion error in exp_bin (sqlsmith) * 6426: Assertion error in rel_find_exp_ (sqlsmith) * 6427: Assertion error in eq_typeswitchloop (sqlsmith) * 6428: Sqlitelogictest crash in aggregation query * 6430: Assertion raised in another eq_typeswitch error (sqlsmith) * 6431: Sqlitelogictest crash in aggregation query with a long having clause * 6432: Assertion error in exp_bin (sqlsmith) * 6433: Sqlitelogictest crash in complex SELECT query with IN operator * 6435: Sqlitelogictest crash in simple select query * 6437: System schemas "profiler" and "json" shouldn't be allowed to be dropped. * 6438: Implement functionality to enforce the restrict option in: DROP SCHEMA xyz RESTRICT; * 6440: Faulty plan generated. Query returns more rows than expected or existing in the view sys.tables.

1 0

IMPORTANT: serious possibility for database corruption
by Sjoerd Mullender 23 Oct '17

23 Oct '17

We just found (and fixed) a serious problem in the Jul2017 releases that can result in database corruption. So far we have only seen the corruption occur after a database (mserver5) restart. We have not yet seen a problem with a server that continues running (but unfortunately no guarantees). We will create a new release candidate with this bug fixed as soon as possible (that will be version 11.27.9, i.e. anything in the 11.27.X series older than this is affected). Currently known work around is to *not* restart the server, but keep it running, and creating a dump of the database using msqldump. This dump will then have to be restored into a fresh database in a server that is not affected by the bug (Dec2016 release or the upcoming Jul2017-SP2). -- Sjoerd Mullender

1 0

Announcement: New Jul2017-SP1 Bugfix release of MonetDB suite
by Sjoerd Mullender 02 Aug '17

02 Aug '17

The MonetDB team at CWI/MonetDB BV is pleased to announce the Jul2017-SP1 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <http://www.monetdb.org/>. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Jul 2017-SP1 bugfix release Build Environment * The Debian and Ubuntu installers have been fixed: there was a file missing in the Jul2017 release. * Added a new RPM called MonetDB-selinux which provides the SELinux policy required to run MonetDB under systemd, especially on Fedora 26. * The Windows installers (*.msi files) are now created using the WiX Toolset. * The Windows binaries are now built using Visual Studio 2015. Because of this, you may need to install the Visual C++ Redistributable for Visual Studio 2015 before being able to run MonetDB. Merovingian * monetdbd was leaking open file descriptors to the mserver5 process it started. This has been fixed. MonetDB Common * Many functions in GDK are now annotated with the GCC attribute __warn_unused_result__ meaning that the compiler will issue a warning if the result of the function (usually an indication of an error) is not used. Bug Fixes * 6325: Merge table unusable in other connections * 6328: Transactional/multi-connection issues with merge tables * 6336: VALUES multiple inserts error * 6339: Mserver5 crashes on nested SELECT * 6340: sample operator takes effect after the execution of the query, expected before * 6341: MERGE TABLE issue: Cannot register * 6342: MERGE TABLE issue: hang * 6344: Spurious errors and assertions (SQLsmith) * 6375: MAL profiler truncates JSON objects larger than 8192 characters

1 0

Announcement: New Jul2017 Feature release of MonetDB suite
by Sjoerd Mullender 28 Jul '17

28 Jul '17

The MonetDB team at CWI/MonetDB BV is pleased to announce the Jul2017 feature release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <http://www.monetdb.org/>. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Jul 2017 feature release MonetDB5 Server * The "sub" prefix of many functions, both at the MAL and the C level, has been removed. * Changed the interfaces of the AUTH* functions: pass values, not pointers to values. * Removed calc.setoid(). * group.subgroup is now called group.group if it is not refining a group. Both group.group and group.subgroup now also have variants with a candidate list. * The allocation schemes for MAL blocks and Variables has been turned into block-based. This reduces the number of malloc()/free() calls. . * Added a new server-side protocol implementation. The new protocol is backwards compatible with the old protocol. Clients can choose whether they want to use the old or the new protocol during the initial handshake with the server. The new protocol is a binary column-based protocol that is significantly faster than the old protocol when transferring large result sets. In addition, the new protocol supports compression using Snappy or LZ4. * Moved the sphinx extension module to its own repository. See https://dev.monetdb.org/hg/MonetDB-sphinx/. * Removed GSL module: it's now a separate (extension) package. See https://dev.monetdb.org/hg/MonetDB-gsl/. * The PCRE library is now optional for systems that support POSIX regular expressions. * Added 5 new sys schema tables: function_languages, function_types, key_types, index_types and privilege_codes. They are pre-loaded with static content and contain descriptive names for the various integer type and code values. See also sql/scripts/51_sys_schema_extension.sql Merovingian * Added handling of a dbextra property per database at the daemon level. The user can set the dbextra property for a database using the command: $ monetdb set dbextra=<path> <database> and the daemon will make sure to start the new server using the correct --dbextra parameter. Client Package * The mclient and msqldump programs lost compatibility with old mserver5 versions (pre 2014) which didn't have a "system" column in the sys.schemas table. * The mclient and msqldump programs lost compatibility with ancient mserver5 versions (pre 2011) which didn't have the sys.systemfunctions table. * Removed the "array" and "quick" functions from the mapi library. To be precise, the removed functions are: mapi_execute_array, mapi_fetch_field_array, mapi_prepare_array, mapi_query_array, mapi_quick_query, mapi_quick_query_array, and mapi_quick_response. * Added a more elaborate \help command for SQL expressions. MonetDB Common * Improved error checking in the logger code (dealing with the write-ahead log); changed return types a several functions from int to gdk_return (i.e., they now return GDK_SUCCEED or GDK_FAIL). The logger no longer calls GDKfatal on error. Instead the caller is responsible for dealing with errors. * BATsort may now create an order index as a by product. * Quantile calculations now use the order index if available (and use BATsort otherwise, producing an order index). * Quantiles calculate a position in the sorted column. If this position is not an integer, we now choose the nearest position, favoring the lower if the distance to the two adjacent positions is equal (round down to nearest integer). * Removed function BATprintf. Use BATprint or BATprintcolumns instead. * Removed BATsave from the list of exported functions. * Replaced BBPincref/BBPdecref with BBPfix/BBPunfix for physical reference count and BBPretain/BBPrelease for logical reference count maintenance. * Removed automatic conversion of 32-bit OIDs to 64 bits on 64-bit architectures. * Removed functions OIDbase() and OIDnew(). * Removed talign field from BAT descriptor. * BATappend now takes an optional (NULL if not used) candidate list for the to-be-appended BAT. * New function BATkeyed(BAT *b) that determines (possibly using a hash table) whether all values in b are distinct. SQL * Made the operator precedence of % equal to those of * and /. All three are evaluated from left to right. * Removed table sys.connections. It was a remnant of an experimental change that had already been removed in 2012. * Protect against runaway profiler events If you hit a barrier block during profiling, the JSON event log may quickly become unwieldy. Event production is protected using a high water mark, which ensures that never within the single execution of MAL block the instruction causes excessive event records. Bug Fixes * 3465: Request: add support for CREATE VIEW with ORDER BY clause * 3545: monetdb commands don't work with -h -P -p options (locally and remotely) * 3996: select * from sys.connections always returns 0 rows. Expected to see at least one row for the active connection. * 6187: Nested WITH queries not supported * 6225: Order of evaluation of the modulo operator * 6289: Crashes and hangs with remote tables * 6292: Runaway SQL optimizer in too many nested operators * 6310: Name resolution error (sqlsmith) * 6312: Object not found in LIMIT clause (sqlsmith) * 6313: Null type resolution in disjunction fails (sqlsmith) * 6319: Server crash on LATERAL (sqlsmith) * 6322: Crash on disjunction with LIMIT (sqlsmith) * 6323: Deadlock calling sys.bbp() * 6324: Sqlitelogictest crash in a IN query (8th) * 6327: The daemon does not respect the actual name of the mserver5 executable * 6330: Sqlitelogictest crash on a complex SELECT query * 6331: sys.statistics column "nils" always contains 0. Expected a positive value for columns that have one or more nils/NULLs * 6332: Sqlitelogictest crash related to an undefined MAL function

1 1

Announcement: New Dec2016-SP5 Bugfix release of MonetDB suite
by Sjoerd Mullender 12 Jun '17

12 Jun '17

The MonetDB team at CWI/MonetDB BV is pleased to announce the Dec2016-SP5 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <http://www.monetdb.org/>. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Dec 2016-SP5 bugfix release Bug Fixes * 6290: Crash (and assertion failure) with a correlated subquery with NOT IN in the select-list * 6291: crash if executes a function with sample operator * 6294: Sqlitelogictest crash * 6296: Another sqllitelogictest crash :( * 6297: 7th sqllitelogictest crash :( * 6300: Protect against missing BATs (sqlsmith) * 6314: Lateral crash report (sqlsmith) * 6315: Exist operator on type bigint missing (sqlsmith) * 6316: Coalesc and limit error (sqlsmith) * 6317: Two-column integer aggregation extremely slow * 6318: Daemon crashes if .merovignian_properties for a database contains more than 42 entries * 6320: The daemon ignores all but the last entry in the .merovignian_properties file when spawning mserver5

1 0