Sorry my bad observation.
The server does crash at points where the clients dumped empty results.
Regards,
Tapomay.
________________________________
From: Tapomay Dey
To: "monetdb-users@lists.sourceforge.net"
Sent: Thursday, July 5, 2012 2:40 PM
Subject: Re: mserver crash under load -16G data, 10 queries each from 2 remote clients
An update:
Tested firing 50 of the same failing query from a single machine.
mserver didn't crash. However some of the clients ended without dumping the results.
Rest of the clients completed successfully taking time from 8 to 200 seconds.
Why does it crash with 2 machines for just 20 queries?
Regards,
Tapomay.
________________________________
From: Tapomay Dey
To: "monetdb-users@lists.sourceforge.net"
Sent: Thursday, July 5, 2012 12:58 PM
Subject: mserver crash under load -16G data, 10 queries each from 2 remote clients
I have a database with Single table. 15 million rows. 16G of data. 58 columns.
MonetDB stores it consuming 32G of data.
mserver Machine specs:
Main memory: 4G
Processor: Intel(R) Xeon(R) CPU E5645 @ 2.40GHz. Single core.
I am performing a test to see how much time it takes if I fire this query multiple times from different machines.
Case 1: I fire an aggregation query 10 times from a remote machine with 5sec interval between each firing.
The result hs 17000 rows with around 4 MB of data.
Memory usage of mserver process reahes a peak of 73%
mclients finish within 43 to 48 seconds.
Case 2: I do the same as above but now from 2 remote machines simultaneously.
Memory usage of mserver process reahes a peak of 73% again. It keeps fluctuating between 70 and 73%.
But after some time mserver restarts and the clients terminate with all the 10 clients on each of both machines saying:
ERROR = !Connection terminated
MAPI = tapomay@54.251.11.181:3306
ACTION= read_line
QUERY = select id1, id2, id3, id4, id5, count(*), sum(metric1),
avg(metric2), sum(metric3), sum(metric4), sum(metric5), sum(metric6),
sum(metric7), sum(metric8), sum(metric9), sum(metric9),
sum(metric10), sum(metric11), sum(metric12), sum(metric13)
from tablename group by id1, id2, id3, id4, id5;
Each query terminates within 5 to 50 seconds with the above error.
In both cases intial state of mserver consumes around 0.5% memory.
Also tested the same under other competing columnar DBs.
They complete the case 2 in around 350 or more seconds average.
They complete case 1 in 200 seconds average.
Following is a last few lines of tail of merovingian.log
2012-07-05 06:30:42 MSG merovingian[31153]: proxying client 1.compute.amazonaws.com:45476 for database 'tablename' to mapi:monetdb:/$
2012-07-05 06:30:42 MSG merovingian[31153]: target connection is on local UNIX domain socket, passing on filedescriptor instead of proxying
2012-07-05 06:30:45 MSG merovingian[31153]: proxying client 2.amazonaws.com:46225 for database 'tablename' to mapi:monetdb:///mnt/tapomay$
2012-07-05 06:30:45 MSG merovingian[31153]: target connection is on local UNIX domain socket, passing on filedescriptor instead of proxying
2012-07-05 06:30:47 MSG merovingian[31153]: proxying client 1.compute.amazonaws.com:45477 for database 'tablename' to mapi:monetdb:/$
2012-07-05 06:30:47 MSG merovingian[31153]: target connection is on local UNIX domain socket, passing on filedescriptor instead of proxying
2012-07-05 06:30:50 MSG merovingian[31153]: proxying client 2.amazonaws.com:46226 for database 'tablename' to mapi:monetdb:///mnt/tapomay$
2012-07-05 06:30:50 MSG merovingian[31153]: target connection is on local UNIX domain socket, passing on filedescriptor instead of proxying
2012-07-05 06:30:52 MSG merovingian[31153]: proxying client 1.compute.amazonaws.com:45478 for database 'tablename' to mapi:monetdb:/$
2012-07-05 06:30:52 MSG merovingian[31153]: target connection is on local UNIX domain socket, passing on filedescriptor instead of proxying
2012-07-05 06:30:55 MSG merovingian[31153]: proxying client 2.amazonaws.com:46241 for database 'tablename' to mapi:monetdb:///mnt/tapomay$
2012-07-05 06:30:55 MSG merovingian[31153]: target connection is on local UNIX domain socket, passing on filedescriptor instead of proxying
2012-07-05 06:30:59 MSG merovingian[31153]: database 'tablename' (1048) was killed by signal SIGKILL
Is there a config that would ask mserver to not kill itself and keep processing at the cost of query execution time.
A few more tens of seconds won't harm much.
Thanks and Regards,
Tapomay.
