Re: High Memory Usage - Sever Not Releasing Resources?

Hello Adam, First of all: - does this also happen with Mar2018? - Is this repeatable? If answers to both questions are Yes, can you please provide us a script/test programme/etc to reproduce the problem? Regards, Jennie

On 8 Jun 2018, at 17:03, Doherty, Adam wrote:

Hi all:

I ran into an issue yesterday when I ran a simple group count query - The query took 7 minutes, 13 seconds to run.

Query was as follows -

SELECT zone, count(*) FROM chronicle.events WHERE event_type = 'mb' GROUP BY zone

This table, which a merge table currently has 546 million events. The merge table is made up of currently two merge table each representing a year (events_2018, and events_2017). Those merge tables are each made up of four tables, events_year_zone_(1-4)

I have a scheduled task which inserts events into the relevant tables every 5 minutes from 5 Microsoft SQL Servers. Roughly 12,000 events broken across 5 inserts (one for each server).

The import process is handled with a C# Windows service that utilizes ODBC to talk to MonetDB.

The server is currently MonetDB v11.27.11 (July2017-SP3) running on VMware 6.5 in a CentOS 7 VM with 64GB of RAM, and 16 CPU cores. When I ran into this issue yesterday, the mserver5 process was utilizing 90% of available RAM.

This was making my insertion task take longer and longer each time, and simple queries like the one above took minutes. Logged into the server via SSH, the input lag as a result of the almost total resource usage caused me to reboot the server.

I restarted the VM, and the same query took 20 seconds.

I'd like to sort where this issue might be coming from, but I'm not sure where to look. This server is currently part of a proof-of-concept project that will soon be going to production.

Any guidance is greatly appreciated.

Adam

_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list