- users-list - monetdb.org

DBSeeder Version 3.0.1
by Walter Weinmann 03 Aug '21

03 Aug '21

We are pleased to release version 3.0.1 of DBSeeder ( https://github.com/KonnexionsGmbH/db_seeder) The most important enhancement is the possibility to deactivate the constraints of the type FOREIGN, PRIMARY and UNIQUE KEY before the DML statements and to activate them again afterwards. The JDBC DatabaseMetaData class is used for this purpose. DBSeeder also supports the latest version of MonetDB available on Docker Hub (DBMS Oct2020-SP5) and Maven Repository (JDBC 3.1.jre8).

1 0

Unable to connect to Monetdb after Upgrade : Challenge string is not valid, it is empty
by AUBERT Simon 02 Jul '21

02 Jul '21

Hello all I had an old monetdb installation on my Windows laptop -Did the upgrade in Oct2020-SP5 Bugfix Release (11.39.17) It really looks like I can start the server C:\Program Files\MonetDB\MonetDB5\bin>mserver5 # MonetDB 5 server v11.39.17 (Oct2020-SP5) # Serving database 'demo', using 4 threads # Compiled for amd64-pc-windows-msvc/64bit # Found 15.862 GiB available main-memory of which we use 12.928 GiB # Copyright (c) 1993 - July 2008 CWI. # Copyright (c) August 2008 - 2021 MonetDB B.V., all rights reserved # Visit https://www.monetdb.org/ for further information # Listening for connection requests on mapi:monetdb://localhost:50000/ # MonetDB/GIS module loaded # MonetDB/SQL module loaded # MonetDB server is started. To stop server press Ctrl-C. But when I want to use the mclient, it asks the user and password and then... nothing and after a few minutes I have this message Challenge string is not valid, it is empty Same thing with an ODBC connection : after a long time, it fails. I have reinstalled Monetdb but nothing changed. I have no idea of what's wrong. Best regards, Simon Simon AUBERT [cid:image001.jpg@01D76D97.13BF6A80] Bourse Maritime - 1, place Lainé - 33 000 Bordeaux Mob : +33(0)6 63 83 76 29

2 2

Re: Re: RSS memory
by Vojtech Kurka 21 Jun '21

21 Jun '21

Hello Roberto, Sjoerd, I think I am hitting the same problem described in this thread. I am running MonetDB in Docker container on a server with 30GB of memory, allocating 21GB of it to MonetDB container. Sometimes, under heavier load I get OOM - process mserver5 uses more memory than is the memory.limit_in_bytes for a cgroup of this container. It looks like this: [Mon Jun 21 12:56:36 2021] memory: usage 20971484kB, limit 20971520kB, failcnt 48706810 [Mon Jun 21 12:56:36 2021] memory+swap: usage 0kB, limit 9007199254740988kB, failcnt 0 [Mon Jun 21 12:56:36 2021] kmem: usage 143696kB, limit 9007199254740988kB, failcnt 0 .. [Mon Jun 21 12:56:36 2021] Tasks state (memory values in pages): [Mon Jun 21 12:56:36 2021] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name [Mon Jun 21 12:56:36 2021] [ 26927] 0 26927 1371 151 53248 0 0 run.sh [Mon Jun 21 12:56:36 2021] [ 27054] 0 27054 134763 326 139264 0 0 monetdbd [Mon Jun 21 12:56:36 2021] [ 27068] 0 27068 40646510 5209845 64159744 0 0 mserver5 [Mon Jun 21 12:56:36 2021] [ 27289] 0 27289 1019 69 45056 0 0 tail [Mon Jun 21 12:56:36 2021] [ 29503] 0 29503 6425 547 86016 0 0 mclient [Mon Jun 21 12:56:36 2021] oom-kill:constraint=CONSTRAINT_MEMCG,nodemask=(null),cpuset=d37591a31d4d010a7b4e34ebb19ba11220a84a264733f70a00ee92899c7918fc,mems_allowed=0,oom_memcg=/docker/d37591a31d4d010a7b4e34ebb19ba11220a84a264733f70a00ee92899c7918fc,task_memcg=/docker/d37591a31d4d010a7b4e34ebb19ba11220a84a264733f70a00ee92899c7918fc,task=mserver5,pid=27068,uid=0 [Mon Jun 21 12:56:36 2021] Memory cgroup out of memory: Killed process 27068 (mserver5) total-vm:162586040kB, anon-rss:20823812kB, file-rss:15568kB, shmem-rss:0kB, UID:0 pgtables:62656kB oom_score_adj:0 Have you made any progress in resolving the OOMs? Things I am trying to test are (I am using docker-compose v2 for running the container, so settings are from there): mem_limit: 21G # this sets memory.limit_in_bytes for the cgroup of the container mem_reservation: 14G # this sets memory.soft_limit_in_bytes oom_score_adj: -1000 # to lower the chance OOM will kill this process Exploring "select * from env();" in MonetDB tells me that the gdk_mem_maxsize is 12251394211, which corresponds to the 81.5% of the mem_reservation, or memory.soft_limit_in_bytes. I am monitoring memory usage of the container and it still is spiking at 19G, which is uncomfortably high, very close to the hard limit of the cgroup. Is this normal? FYI, originally meant only for PostgreSQL, but it became our standard - we are using vm.overcommit = 2 and vm.overcommit_ration = 100, if it changes anything. I would be thankful for any clue on how to run MonetDB in the container reliably as right now it is causing me quite a headache. Running it on a host solely for the MonetDB would be a last resort for me. Thank you and have a great day!

2 1

New MonetDB JDBC driver released
by Sjoerd Mullender 16 Jun '21

16 Jun '21

The MonetDB team at MonetDB BV is pleased to announce a bugfix release of the MonetDB Java jars and JDBC driver. We recommend all users of MonetDB JDBC driver to upgrade to version: monetdb-jdbc-3.1.jre8.jar The download location is as usual: <https://www.monetdb.org/downloads/Java/>. For a list of changes please read: <https://www.monetdb.org/downloads/Java/ChangeLog>. For information and tips on using the MonetDB JDBC driver, see: <https://www.monetdb.org/downloads/Java/release.txt>. Note: as of release monetdb-jdbc-3.0.jre8.jar the MonetDB JDBC Driver class name has been changed to: org.monetdb.jdbc.MonetDriver It used to be nl.cwi.monetdb.jdbc.MonetDriver which is now marked as deprecated. Please use the new driver class name asap in your configuration files/scripts and/or Java code. The old driver class name is still included in the jar file to ease the transition for existing deployments. However it will be removed in a future release of the MonetDB JDBC driver. -- Sjoerd Mullender

1 0

DBSeeder Version 2.9.1
by Walter Weinmann 12 Jun '21

12 Jun '21

The latest version of DBSeeder is now available on GitHub: https://github.com/KonnexionsGmbH/db_seeder. This also includes from MonetDB the latest available versions on Docker Hub (Oct2020-SP5) and https://www.monetdb.org/downloads/Java (JDBC monet-jdbc-3.0.jre8). MonetDB now shows a huge performance improvement after a manual commit (instead of autocommit) is executed in DBSeeder after each batch of insert operations. [image: Company_9.9.9_win10.png]

1 0

Linux kernel vm settings
by Roberto Cornacchia 06 Jun '21

06 Jun '21

Hi there, I looked around but couldn't find any recommendation about kernel vm settings in Linux for MonetDB. In particular: - vm.overcommit_memory: 0 (default) : a heuristics decides whether overcommitting is allowed 1: no check, overcommit is always allowed 2: overcommitting is regulated by vm.overcommit_ratio (default = 50%) Do I understand correctly that using vm.overcommit_memory=1 will only make the OOM kill mserver5 when the total VM available is exhausted? If that is true, should it be reasonably safe to use on a server that is mainly intended for MonetDB, as long as sufficient disk space is available? - vm.swappiness Generic recommendations are usually 60 for a desktop and 30 for a server. Oracle recommends 10. Redis recommends 1. Are there studies / recommendations for MonetDB?

5 8

DBSeeder Version 2.9.0
by Walter Weinmann 05 Jun '21

05 Jun '21

We are happy to announce that version 2.9.0 of DBSeeder ( https://github.com/KonnexionsGmbH/db_seeder) is now available for general use. DBSeeder also supports the latest version of MonetDB available on DockerHub (Oct2020-SP5).

1 0

for the Cyrillic alphabet, lower does not work
by Илья Хайбуллин 17 May '21

17 May '21

2 3

gcc optimization level?
by Roberto Cornacchia 14 May '21

14 May '21

I was wondering, -DCMAKE_BUILD_TYPE=Release (or RelWithDebInfo) doesn't seem to enable -O2 on gcc. Is there a good reason for it or was it just forgotten?

2 2

Announcement: New Oct2020-SP5 Bugfix release of MonetDB suite
by Sjoerd Mullender 10 May '21

10 May '21

The MonetDB team at MonetDB BV is pleased to announce the Oct2020-SP5 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <https://www.monetdb.org/>. For details on this release, please see the release notes at <https://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <https://www.monetdb.org/downloads/>. Oct2020-SP5 Bugfix Release (11.39.17) Windows Packaging * Added the monetdbe library to the Windows installer. MonetDB Common * A bug that would very occasionally produce an error "strPut: incorrectly encoded UTF-8", even when no incorrectly coded characters are used at all, has been fixed. It was the result of a rare combination of strings having been added to the column that caused essentially an off-by-one type of error to occur. SQL Frontend * A bug where a sequence of TRUNCATE TABLE and COPY INTO the just truncated table would result in success being reported to both queries, but the table still being empty afterwards, has been fixed. Merovingian * When stopping monetdbd using the `monetdbd stop' command, this command now waits for 5 seconds longer than the exittimeout value before it kills the monetdbd daemon instead of only 30 seconds total (or until that daemon stops earlier). This gives the daemon enough time to terminate the mserver5 processes that it is managing. If exittimeout is negative, the daemon and the monetdbd process initiating the stop wait indefinitely until the mserver5 processes have stopped. Bug Fixes * 3336: DB files not removed if all rows are deleted, even after restart * 7104: Monetdbe NTILE function does not produce correct ordering * 7108: Monetdb crashes on query execution * 7109: MERGE Statement incorrectly reports that input relation matches multiple rows * 7110: Monetdb Query parsing consistency issues in the latest release (Remote Table)

1 0