February 2010

NOTE: Before upgrading to the latest “Feb2010” release from a release of MonetDB older than the Nov2008 release, please make a dump of your database, and rename/move/backup the database directories (‘dbfarm’, ‘sql_logs’, ‘xquery_logs’). After the upgrade, restore your database, again. Once you have checked and confirmed the correctness of the restore, you can delete the above renamed/moved/backed-up old database directories.

When upgrading from a release of MonetDB older than the May2009 release, you need to move your write-ahead log following the “recommended” instructions for SQL and/or XQuery.

MonetDB Common

• Various concurrency bugs were fixed.
• Various changes were made to run better on systems that don’t have enough memory to keep everything in core that was touched during query processing. This is done by having the higher layers giving hints to the database kernel about future use, and the database kernel giving hings to the operating system kernel about how (virtual) memory is going to be used.
• Fixed bug in mergejoin implementation. This fixes bug #2952191.
• Added support for compiling on Windows using the Cygwin-provided version of flex.
• Fix compilation issue when configured with –with-curl. This fixes bug #2924999.
• Added implementation of MT_getrss() for Solaris. This yields in the kernel knowing about its (approximate) memory usage to try and help the operating system to free that memory that is best to free, instead of a random page, e.g. the work of the vmtrim thread.
• Implemented a “fast” string BAT append: Under certain conditions, instead of inserting values one-by-one, we now concatenate the string heap wholesale and just manipulate the offsets. This works both for BATins and BATappend.
• Changed the string heap implementation to also contain the hashes of strings.
• Changed the implementation of the string offset columns to be dynamically sized.

MonetDB Server 5

• The MonetDB 5 code base underwent a series of small/medium changes:
• Solving name conflicts with external libraries in mal_box.
• Dependency graph generation in MAL debugger fixed.
• Code hardening against out of memory errors and based on Coverity checks.
• Recycler improved to deal with SQL plans from different sessions.
• Profiler extended to report the argument types, user, and thread id.
• MAL interpreter reports an event before and after the instruction.
• Dataflow also allowed for updates on temporary BATs.
• Reorder optimizer better respects the dataflow.
• All update instructions return their target to mark the dataflow.
• Clean up of (bat)calc module.
• Packing pieces together simplified and sped up.
• The MAL interpreter has been extended with an operator admission scheme, which is active during parallel execution. It blocks threads if the total amount of memory needed for the operator can not be claimed. Only if there is one operation left to execute, it won’t block. The admission level is controlled by a threshold, which is set to 90% of the physical memory.
• The ilike[u]select operations has been included in the repertoire recognized by the mergetable optimizer for push through of selections.
• The joinpath optimizer has been extended with recognition of the leftjoin. This way, series of leftjoin operations can be optimized by looking at the smallest starting point.
• Introduced a dictionary encoding option to the optimizer pipeline. Encoding is initiated with a SQL call compress(tablename).
• Removed the ability to redirect to other running databases in the same dbfarm. This functionality has been taken over completely by merovingian, and only results in confusement these days, bug #2891191.
• Added –enable-console configure argument, defaulting to ‘yes’ for now. Disabling the server console increases security by avoiding local access exploits. This is not the default since our Testing setup cannot deal with a console-less server yet.
• Report detected amount of main memory and cpu cores in output of mserver5 --version.

Client Packages

• Fixed a bug in the Mapi libary when sending very large queries to the server.
• Implemented BLOB and CLOB support in ODBC driver.
• Fixed crash upon connect to a server that is under maintenance but has the requested language (scenario) not loaded.
• Fixed bug in the ODBC driver where a non-ASCII character at the end of a string caused an error when the string was converted to wide characters.
• mclient timer information (-t option) is now written to standard error instead of standard output. This makes getting timer information easier when there is a large amout of regular output (which can be redirected to a file or /dev/null).
• Implemented new function mapi_fetch_field_len() which returns the length (excluding trailing NULL byte) of the field returned by mapi_fetch_field().

The following bug tracker items were fixed since the previous “Nov 2009” release: The detailed list of changes can be found in the source code CVS logs.