Hi all,
I'm using monetdb on two servers and meet some strange behavior.
The two servers are of the same configuration as the following:
CPU: Intel(R) Xeon(R) CPU E5-2430L, 6 Cores, 12 Threads, 2 GHz, 64
bits.
Memory: 64GB
OS: Ubuntu Server 13.04 with 3.8.0-25-generic x86_64 kernel
MonetDB: v11.15.7
And they are both bought for less than 4 months.
The problem is that the performance of MonetDB is very unstable, even
for very simple select * queries.
The experiment setting is as the following:
Data:
Server1: Table1 loaded from a 19GB CSV file (CSV1).
Server2: Table1 loaded from another 19GB CSV file (CSV2).
CSV1 and CSV2 have the same schema but have different contents.
But they both have size of about 19GB and both have 155000000 tuples.
The schema is:
Field1 VARCHAR(16),
Field2 VARCHAR(100),
Field3 DATE,
Field4 FLOAT,
Field5 VARCHAR(64),
Field6 CHAR(3),
Field7 CHAR(6),
Field8 VARCHAR(32),
Field9 INT
In addition, both the tables have indices on Field1, Field2 and Field3.
Experiment Query:
echo -e '\\w-1 \n \\> /dev/null \n select * from Table1;' |
mclient -d expdatabase --interactive
The problem is that the performance varies a lot between the servers and
also
between different runs on the same server.
Here is the result:
Server1: 4m 12s
Server2: 305m 29s
The system conditions during the experiment:
Disk speed:
I run the command: cat CSV > /dev/null. The speed
is normal, about 2m ~ 2m 30s .
System load:
The machine load during the query executions are both very low.
The CPUs are almost completely idle and free memories are abundant.
Each monetdb server took about 50G.
To eliminate possible system configuration problems. I did two more
experiments.
1) I move CSV2 to server1, stop the server1
monetdb and build and start a new monetdb database using CSV2.
Then I run the same query again on the new database. The time
elapse is 134m 8s.
2) After 1) is done I stop the new monetdb and restart the old monetdb
on server1, and run the same query again. This time the time
elapse becomes 388m 37s (Comparing 4m 12s in the first run).
Further disk observations:
During the above two experiments, I monitor the disk read speed of
monetdb using iotop. The speed keeps about only 500 ~ 550 k/s. While the
read speed of cat Table1.csv > /dev/null keeps around 135m/s.
I did not do any configuration about monetdb. Is it possible that I
misconfigured something?
Thanks.
Victor
