May 2009

MonetDB May2009-SP2 release

Release “May2009-SP2” consists of the following updated components

IMPORTANT NOTE: Before upgrading to the latest “May2009-SP2” release from a release of MonetDB older than the Nov2008 release, please make a dump of your database, and rename/move/backup the database directories (‘dbfarm’, ‘sql_logs’, ‘xquery_logs’). After the upgrade, restore your database, again. Once you have checked and confirmed the correctness of the restore, you can delete the above renamed/moved/backed-up old database directories.

May 2009 SP2 bug-fix release

This is the second bug fix release of the May2009 version. This version can be installed “on top of” the May2009 release. The database format is fully compatible, so there is no need to do a dump and restore. (Although it is always a good idea to make backups.)

Some of the bugs that were fixed:

• Fixed a problem where in certain condition you could get an error on Windows saying “GDKremovedir: rmdir(bat\DELETE_ME) failed”.
• Various improvements to the Python and Ruby clients.
• Fix mserver5 crash when started on demand by merovingian (SF bug #2805340).
• Check for write errors during dump and abort when they occur.
• Some innocuous looking XQuery queries could produce large incorrect results and take a relatively long time to do so. SF bug #2815820.
• Fixed some potential buffer overflow problems in the Mapi library.
• Improved help message from mclient (SF bug 2818536).
• Fixed certain SQL LIKE queries on some O/S.
• Fixed SQL dump in mclient and msqldump to dump values of some less used types correctly.
• The Debian/Ubuntu installation now also contains the MonetDB PHP client support files.
• Fixed several performance issues (among others: SF bug 2821251).
• The various client libraries now also recognize the wrd type which may be emitted by the SQL server.
• In SQL, the char and varchar types now use Unicode code points (characters to you and me) as the unit for counting instead of bytes in the UTF-8 representation (SF bug 2822855).
• Fixed some UI issues with the XQuery administrator console.
• Fixed a bug in SQL when handling IN and OR lists with equal values.
• Fixed a stack overflow condition in SQL with certain types of quite complex queries (SD bug 2827784).

The following bug tracker items were fixed since the May2009 SP1 release:

• [ 1840239 ] BAT capacity checking needed
• [ 2805340 ] Merovingian (?): mserver5 segfaults if started by a client
• [ 2815820 ] PF: Endless loop in innocent xquery
• [ 2818536 ] improve mclient -u / -p help
• [ 2818931 ] MAL: segv on mortal soul’s syntax error
• [ 2819972 ] function problom —-return and elseif
• [ 2821251 ] PF: Complex optimization overly complex
• [ 2822855 ] utf8 real size upon insert

May 2009 SP1 bug-fix release

This is a bug fix release of the May2009 version. This version can be installed “on top of” the May2009 release. The database format is fully compatible, so there is no need to do a dump and restore. (Although it is always a good idea to make backups.)

Some of the bugs that were fixed:

• mclient and msqldump would cache all results. This is fixed.
• The Fedora RPM for the Ruby clients package is now included.
• Some files were missing in the Python client directory. They are now included in the client source tar ball.
• mclient now shows the results of all subqueries in a query batch that was passed on the command line using the -s option if there was an error in one of them.
• mclient \d and \D interactive commands now obey the | command.
• Fixed naming of the document and collection when both the –input and –collection arguments are given to mclient (XQuery).
• Various fixes to the low level memory mapped file support.
• Fixes to the geom module.

The following bug tracker items were fixed since the May2009 release:

• [ 2603876 ] XQ: fn:put() tests segfault with ALG back-end
• [ 2687549 ] SQL: test type_dump_test.SF-989257.SQL.sh fails since Feb 28
• [ 2737889 ] rand() is not executed for every row when using alias
• [ 2782202 ] SQL: dump&restore fully usable only by admin
• [ 2791356 ] Floating point exception
• [ 2797767 ] mclient –dump caches dump
• [ 2803477 ] alter table drop contraint error
• [ 2804823 ] The Mapi Client Utility
• [ 2805513 ] fn:trace() produces output and error
• [ 2807038 ] tab + ) + mal function = seg fault

Windows The Windows binaries for SQL and ODBC were built using Microsoft Visual Studio 2008, unlike previous releases which were built using Microsoft Visual Studio 2005. The main implication of this change is that the ODBC driver should now also install on Windows Vista.

The Perl, PHP, and Python clients were removed from the Windows installers. The clients are still available in the source distribution (although the Python client has changed significantly).

Fedora There is no longer an RPM for MonetDB-client-ruby. In order to update to this release, you will first have to remove MonetDB-client-ruby after which you can update to this version. (A new version of the RPM will probably appear in a future release.)

Ubuntu The Ubuntu version was built for Ubuntu 9.04, the Jaunty Jackalope. The Feb2009 release was built for Ubuntu 8.10, the Intrepid Ibex.

MonetDB Common< The way string data is stored in the database has changed for systems with a 64-bit word size and 32-bit OIDs. When the server first opens a database that was created using the old format, the database is updated automatically. This change affect only the 64-bit Windows build. The other 64-bit builds that are provided by us use 64-bit OIDs and are unaffected by the change.

MonetDB Server 5 Parallel loading has been disabled since there was an inherent problem with the implementation. This also affects SQL.

SQL

Write-ahead log

The default location of the write-ahead log has changed. At startup, when the server sees there is an SQL catalog but no write-ahead log, the server will exit immediately with an error message. You can do one of two things (not counting deleting your whole database):

• move the log files to the new location (recommended): create the directory that is mentioned by the error message and move the log files (two files, log and log._n_ where _n_ is some integer) there;
• start the server with the --set sql_logdir=... option (where you replace ... with the appropriate path) to point the server to the old location.

In either case you need to know the old location of the write-ahead logs. This old location was _$prefix_/var/MonetDB5/sql_logs/_<dbname>_/sql/ where _$prefix_ is the prefix under which MonetDB was installed and _<dbname>_ the database name. The error message mentioned above gives the expected location of the write-ahead logs.

Note that when you use the Windows installer and you start the server using the Start menu, the write-ahead log is moved to the new location automatically.

Authentication The authentication mechanism has changed somewhat so that it is now possible to dump and restore password information. Note that the passwords themselves cannot be retrieved: the dump/restore mechanism uses a hash of the password.

Merovingian The following features were implemented in the merovingian and monetdb programs:

• The logging of merovingian has changed. TME records are no longer issued, instead each line is prefixed with a timestamp, syslog style. Feature Request #2782776
• merovingian now reopens its logfiles when being sent the SIGHUP signal. This allows utilities like e.g. logrotate to rotate the merovingian.log file. Feature Request #2782776
• The merovingian default log file location has changed from ‘/var/log/merovingian.log’ into ‘/var/log/MonetDB/merovingian.log’.

XQuery Like the previous Feb 2009 SP2 release, the default Algebra-based Pathfinder compiler ("ALG") still does not support a few XQuery features and extensions. A detailed list and temporary work-arounds by resorting to the deprecated “milprint_summer” ("MPS") version of the Pathfinder compiler can be found [obsolete]

Write-ahead log The default location of the write-ahead log has changed. At startup, when the server sees there is an XQuery catalog but no write-ahead log, the server will exit immediately with an error message. You can do one of two things (not counting deleting your whole database):

• move the log files to the new location (recommended): create the directory that is mentioned by the error message and move the log files (two files, log and log._n_ where _n_ is some integer) there;
• start the server with the --set xquery_logdir=... option (where you replace ... with the appropriate path) to point the server to the old location.

In either case you need to know the old location of the write-ahead logs. This old location was _$prefix_/var/MonetDB4/xquery_logs/_<dbname>_/xquery/ where _$prefix_ is the prefix under which MonetDB was installed and _<dbname>_ the database name. The error message mentioned above gives the expected location of the write-ahead logs.

Note that when you use the Windows installer and you start the server using the Start menu, the write-ahead log is moved to the new location automatically.

Client Packages

Mapi Support for SSL-encrypted connections with the server has been discontinued. The protocol between client and server has changed slightly. This means you cannot use an old client with a new server.

Python The Python client modules were rewritten and are now in pure Python and so do not depend on any other part of MonetDB. The new module has support for Python 2.5, 2.6 and 3.0, and it is Python DBAPI 2.0 compatible.

RubyThe Ruby client modules were rewritten and are now in pure Ruby and so do not depend on any other part of MonetDB.