Hello Roberto, Sjoerd,

I think I am hitting the same problem described in this thread. I am running MonetDB in Docker container on a server with 30GB of memory, allocating 21GB of it to MonetDB container.

Sometimes, under heavier load I get OOM - process mserver5 uses more memory than is the memory.limit_in_bytes for a cgroup of this container. It looks like this:

[Mon Jun 21 12:56:36 2021] memory: usage 20971484kB, limit 20971520kB, failcnt 48706810

[Mon Jun 21 12:56:36 2021] memory+swap: usage 0kB, limit 9007199254740988kB, failcnt 0

[Mon Jun 21 12:56:36 2021] kmem: usage 143696kB, limit 9007199254740988kB, failcnt 0

...

[Mon Jun 21 12:56:36 2021] Tasks state (memory values in pages):

[Mon Jun 21 12:56:36 2021] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name

[Mon Jun 21 12:56:36 2021] [ 26927] 0 26927 1371 151 53248 0 0 run.sh

[Mon Jun 21 12:56:36 2021] [ 27054] 0 27054 134763 326 139264 0 0 monetdbd

[Mon Jun 21 12:56:36 2021] [ 27068] 0 27068 40646510 5209845 64159744 0 0 mserver5

[Mon Jun 21 12:56:36 2021] [ 27289] 0 27289 1019 69 45056 0 0 tail

[Mon Jun 21 12:56:36 2021] [ 29503] 0 29503 6425 547 86016 0 0 mclient

[Mon Jun 21 12:56:36 2021] oom-kill:constraint=CONSTRAINT_MEMCG,nodemask=(null),cpuset=d37591a31d4d010a7b4e34ebb19ba11220a84a264733f70a00ee92899c7918fc,mems_allowed=0,oom_memcg=/docker/d37591a31d4d010a7b4e34ebb19ba11220a84a264733f70a00ee92899c7918fc,task_memcg=/docker/d37591a31d4d010a7b4e34ebb19ba11220a84a264733f70a00ee92899c7918fc,task=mserver5,pid=27068,uid=0

[Mon Jun 21 12:56:36 2021] Memory cgroup out of memory: Killed process 27068 (mserver5) total-vm:162586040kB, anon-rss:20823812kB, file-rss:15568kB, shmem-rss:0kB, UID:0 pgtables:62656kB oom_score_adj:0

Have you made any progress in resolving the OOMs?

Things I am trying to test are (I am using docker-compose v2 for running the container, so settings are from there):

mem_limit: 21G # this sets memory.limit_in_bytes for the cgroup of the container

mem_reservation: 14G # this sets memory.soft_limit_in_bytes

oom_score_adj: -1000 # to lower the chance OOM will kill this process

Exploring "select * from env();" in MonetDB tells me that the gdk_mem_maxsize is 12251394211, which corresponds to the 81.5% of the mem_reservation, or memory.soft_limit_in_bytes.

I am monitoring memory usage of the container and it still is spiking at 19G, which is uncomfortably high, very close to the hard limit of the cgroup. Is this normal?

FYI, originally meant only for PostgreSQL, but it became our standard - we are using vm.overcommit = 2 and vm.overcommit_ration = 100, if it changes anything.

I would be thankful for any clue on how to run MonetDB in the container reliably as right now it is causing me quite a headache. Running it on a host solely for the MonetDB would be a last resort for me.

Thank you and have a great day!