On Jul 09, 2015, at 22:25, Anu Raj Srivastava wrote:

Hi Jennie,

Thanks for your response. Please find responses in red:-

I appreciate your help.

Anu

-----Original Message----- From: developers-list [mailto:developers-list-bounces+anu.srivastava=rms.com@monetdb.org] On Behalf Of Ying Zhang Sent: Wednesday, July 08, 2015 1:18 PM To: Communication channel for developers of the MonetDB suite. Subject: Re: MonetDB causing disk space to fill up

Hello Anu,

First of all. Thanks for trying MonetDB.

Normally, MonetDB tries to clean up things (e.g., remove/discard temporary files, process the Write-Ahead-Log files) at a regular time and/or if there is some idle time. In your case, it could be that the disk is filled before MonetDB got a sufficient chance to do the cleaning. The WAL files can be large and takes some time to process. Would you please provide us some more information, so that we can look further into your issue:

- which monetdb version? We have discovered several leaks in Oct2014, which are fixed in Jul2015 – I have installed package “MonetDB-11.19.15-20150603.el7.centos.x86_64.rpm” which I believe is Oct2014 version but I believe Jul2015 version is not available yet for CentOs at least I can’t find one.

Hai Anu, It’s still an R.C., so the binaries are not available yet. But you can compile it yourself. The nightly-built source tarbal is here: http://monetdb.cwi.nl/testweb/web/56200:2b3e7f31da25/ Here is some help for source compilation: https://www.monetdb.org/Developers/SourceCompile Hope this helps. Regards, Jennie

- which OS? – CentOs 7 - disk size? available disk space? data size? – Since I am running it on Virutal Machine I have given it 50 GB disk space with around 38 GB still available. Data size is around 2 GB.

- what do your queries do? read-only queries? or read-only queries intertwined with updates? how exactly? – For now I am testing against 10 read-only, 9 of them are pure analytic queries which are returning only one row and one last query is returning around 1 million row. So I believe queries are not too crazy - updates frequency? –N/A - In your loop, I assume the same query set is repeated. how many queries are there? – Total of 10 queries

Regards,

Jennie

...

On Jul 08, 2015, at 00:15, Anu Raj Srivastava wrote:

Hi,

I have a bunch of test queries which I am testing against MonetDB. Queries all by itself run fine but if you run the same set of queries in a loop disk runs out of space. I am not sure why would memory map files won't clean up automatically and why do we have to start the server again to clean up these files? Is there a workaround to this problem or is this a known issue in MonetDB?

Thanks, Anu _______________________________________________ developers-list mailing list developers-list@monetdb.org https://www.monetdb.org/mailman/listinfo/developers-list

_______________________________________________ developers-list mailing list developers-list@monetdb.org https://www.monetdb.org/mailman/listinfo/developers-list _______________________________________________ developers-list mailing list developers-list@monetdb.org https://www.monetdb.org/mailman/listinfo/developers-list

On Jul 13, 2015, at 22:32, Anu Raj Srivastava wrote:

Hey Jennie,

I did upgrade to latest version but I am facing the same issue. Further debugging into the issue it seems like it is ConnectionPool issue. When I use new connection for each query MonetDB seems to be able to clean up the resources

Ok, MonetDB is doing what it’s supposed to do.

but if I use ConnectionPool for some reason DiskSpace keep filling up.

If a connection is kept open, I’d expect MonetDB to be less eager to clean up resources. However, we generally recommend using a ConnectionPool, to avoid too much overhead caused by making a new connection for every query. So my suggestion for now would be to keep using the ConnectionPool, but try to find the reason why your disk is filled up.

Do you have any idea why that could happen?

Hmmm, no. We’ve fixed quite some leaks in Jul2015... Some more thinking: - After how many iterations of repeatedly executing the queries is your disk filled up? - Can you please try to run your queries with mclient to observe the behaviour of MonetDB (and disk usage)? @Hannes: sorry to disturb you during holiday. But if possible, would you please have a look at this thread. Does anything ring a bell? Thanks! Regards, Jennie

Thanks, Anu

-----Original Message----- From: developers-list [mailto:developers-list-bounces+anu.srivastava=rms.com@monetdb.org] On Behalf Of Ying Zhang Sent: Friday, July 10, 2015 7:49 AM To: Communication channel for developers of the MonetDB suite. Subject: Re: MonetDB causing disk space to fill up

...

On Jul 09, 2015, at 22:25, Anu Raj Srivastava wrote:

Hi Jennie,

Thanks for your response. Please find responses in red:-

I appreciate your help.

Anu

-----Original Message----- From: developers-list [mailto:developers-list-bounces+anu.srivastava=rms.com@monetdb.org] On Behalf Of Ying Zhang Sent: Wednesday, July 08, 2015 1:18 PM To: Communication channel for developers of the MonetDB suite. Subject: Re: MonetDB causing disk space to fill up

Hello Anu,

First of all. Thanks for trying MonetDB.

Normally, MonetDB tries to clean up things (e.g., remove/discard temporary files, process the Write-Ahead-Log files) at a regular time and/or if there is some idle time. In your case, it could be that the disk is filled before MonetDB got a sufficient chance to do the cleaning. The WAL files can be large and takes some time to process. Would you please provide us some more information, so that we can look further into your issue:

- which monetdb version? We have discovered several leaks in Oct2014, which are fixed in Jul2015 – I have installed package “MonetDB-11.19.15-20150603.el7.centos.x86_64.rpm” which I believe is Oct2014 version but I believe Jul2015 version is not available yet for CentOs at least I can’t find one.

Hai Anu,

It’s still an R.C., so the binaries are not available yet. But you can compile it yourself. The nightly-built source tarbal is here: http://monetdb.cwi.nl/testweb/web/56200:2b3e7f31da25/

Here is some help for source compilation: https://www.monetdb.org/Developers/SourceCompile

Hope this helps.

Regards,

Jennie

...

- which OS? – CentOs 7 - disk size? available disk space? data size? – Since I am running it on Virutal Machine I have given it 50 GB disk space with around 38 GB still available. Data size is around 2 GB.

- what do your queries do? read-only queries? or read-only queries intertwined with updates? how exactly? – For now I am testing against 10 read-only, 9 of them are pure analytic queries which are returning only one row and one last query is returning around 1 million row. So I believe queries are not too crazy - updates frequency? –N/A - In your loop, I assume the same query set is repeated. how many queries are there? – Total of 10 queries

Regards,

Jennie

...

On Jul 08, 2015, at 00:15, Anu Raj Srivastava wrote:

Hi,

I have a bunch of test queries which I am testing against MonetDB. Queries all by itself run fine but if you run the same set of queries in a loop disk runs out of space. I am not sure why would memory map files won't clean up automatically and why do we have to start the server again to clean up these files? Is there a workaround to this problem or is this a known issue in MonetDB?

Thanks, Anu _______________________________________________ developers-list mailing list developers-list@monetdb.org https://www.monetdb.org/mailman/listinfo/developers-list

_______________________________________________ developers-list mailing list developers-list@monetdb.org https://www.monetdb.org/mailman/listinfo/developers-list _______________________________________________ developers-list mailing list developers-list@monetdb.org https://www.monetdb.org/mailman/listinfo/developers-list

_______________________________________________ developers-list mailing list developers-list@monetdb.org https://www.monetdb.org/mailman/listinfo/developers-list _______________________________________________ developers-list mailing list developers-list@monetdb.org https://www.monetdb.org/mailman/listinfo/developers-list

On 19 Jul 2015, at 13:23, Hannes Mühleisen wrote:

Hi,

...

On 16 Jul 2015, at 23:11, Ying Zhang wrote:

On Jul 13, 2015, at 22:32, Anu Raj Srivastava wrote:

...

Hey Jennie,

I did upgrade to latest version but I am facing the same issue. Further debugging into the issue it seems like it is ConnectionPool issue. When I use new connection for each query MonetDB seems to be able to clean up the resources

Ok, MonetDB is doing what it’s supposed to do.

...

but if I use ConnectionPool for some reason DiskSpace keep filling up.

If a connection is kept open, I’d expect MonetDB to be less eager to clean up resources. However, we generally recommend using a ConnectionPool, to avoid too much overhead caused by making a new connection for every query. So my suggestion for now would be to keep using the ConnectionPool, but try to find the reason why your disk is filled up.

...

Do you have any idea why that could happen?

Hmmm, no. We’ve fixed quite some leaks in Jul2015... Some more thinking: - After how many iterations of repeatedly executing the queries is your disk filled up? - Can you please try to run your queries with mclient to observe the behaviour of MonetDB (and disk usage)?

@Hannes: sorry to disturb you during holiday. But if possible, would you please have a look at this thread. Does anything ring a bell? Thanks!

I tested this with ac3p0 connection pool (limited to 4 concurrent connections) and repeating queries from 10 threads (SELECT MIN(something) FROM …). I was not able to observe any (!) increase in storage footprint. So without a minimal reproducible example of Java code + DB schema + sample data, it is hard to tell what is going on. Is this running in auto commit? Using temporary tables?

Small follow-up: When forcing the allocation of a large intermediate (SELECT MIN(something+1) FROM …), additional storage space is indeed claimed. The attached plot shows dbfarm size over time while 10 threads are running 1000 queries each using a 5-connection pool. You can see how the amount of additional storage drops back to exactly where it was once the pool shuts down at around 350s, and that there is no accumulation of additional storage over the course of the pool being active. Hannes

...

...

-----Original Message----- From: developers-list [mailto:developers-list-bounces+anu.srivastava=rms.com@monetdb.org] On Behalf Of Ying Zhang Sent: Friday, July 10, 2015 7:49 AM To: Communication channel for developers of the MonetDB suite. Subject: Re: MonetDB causing disk space to fill up

...

On Jul 09, 2015, at 22:25, Anu Raj Srivastava wrote:

Hi Jennie,

Thanks for your response. Please find responses in red:-

I appreciate your help.

Anu

-----Original Message----- From: developers-list [mailto:developers-list-bounces+anu.srivastava=rms.com@monetdb.org] On Behalf Of Ying Zhang Sent: Wednesday, July 08, 2015 1:18 PM To: Communication channel for developers of the MonetDB suite. Subject: Re: MonetDB causing disk space to fill up

Hello Anu,

First of all. Thanks for trying MonetDB.

Normally, MonetDB tries to clean up things (e.g., remove/discard temporary files, process the Write-Ahead-Log files) at a regular time and/or if there is some idle time. In your case, it could be that the disk is filled before MonetDB got a sufficient chance to do the cleaning. The WAL files can be large and takes some time to process. Would you please provide us some more information, so that we can look further into your issue:

- which monetdb version? We have discovered several leaks in Oct2014, which are fixed in Jul2015 – I have installed package “MonetDB-11.19.15-20150603.el7.centos.x86_64.rpm” which I believe is Oct2014 version but I believe Jul2015 version is not available yet for CentOs at least I can’t find one.

Hai Anu,

It’s still an R.C., so the binaries are not available yet. But you can compile it yourself. The nightly-built source tarbal is here: http://monetdb.cwi.nl/testweb/web/56200:2b3e7f31da25/

Here is some help for source compilation: https://www.monetdb.org/Developers/SourceCompile

Hope this helps.

Regards,

Jennie

...

- which OS? – CentOs 7 - disk size? available disk space? data size? – Since I am running it on Virutal Machine I have given it 50 GB disk space with around 38 GB still available. Data size is around 2 GB.

- what do your queries do? read-only queries? or read-only queries intertwined with updates? how exactly? – For now I am testing against 10 read-only, 9 of them are pure analytic queries which are returning only one row and one last query is returning around 1 million row. So I believe queries are not too crazy - updates frequency? –N/A - In your loop, I assume the same query set is repeated. how many queries are there? – Total of 10 queries

Regards,

Jennie

...

On Jul 08, 2015, at 00:15, Anu Raj Srivastava wrote:

Hi,

I have a bunch of test queries which I am testing against MonetDB. Queries all by itself run fine but if you run the same set of queries in a loop disk runs out of space. I am not sure why would memory map files won't clean up automatically and why do we have to start the server again to clean up these files? Is there a workaround to this problem or is this a known issue in MonetDB?

Thanks, Anu _______________________________________________ developers-list mailing list developers-list@monetdb.org https://www.monetdb.org/mailman/listinfo/developers-list

_______________________________________________ developers-list mailing list developers-list@monetdb.org https://www.monetdb.org/mailman/listinfo/developers-list _______________________________________________ developers-list mailing list developers-list@monetdb.org https://www.monetdb.org/mailman/listinfo/developers-list

_______________________________________________ developers-list mailing list developers-list@monetdb.org https://www.monetdb.org/mailman/listinfo/developers-list _______________________________________________ developers-list mailing list developers-list@monetdb.org https://www.monetdb.org/mailman/listinfo/developers-list