Hi,
I haven't understand it completely, does it mean that it will always use a
single thread while doing Group by.
Also I have noticed one behaviour as below --
1) Very Fast Execution ---
Select product_type,sum(ord_qty) FROM bookings where ord_entry_date
between '2014-04-01' and '2014-10-01' group by product_type;
2) Suddenly very slow execution , the moment I add another AGGREGATE
function something like below --
select product_type,sum(ord_qty) ,COUNT(DISTINCT sales_document) FROM
bookings where ord_entry_date between '2014-04-01' and '2014-10-01' group
by product_type;
So with 2 AGGREGATE functions it behaves very slow as compared to 1
AGGREGATE function.
Also even with 2 AGGREGATE functions example above , if i add more
dimentions and group by 2-3 columns then there is a slight improvement in
performance even though we are adding more GROUP BYs
Is this behaviour is expected, does anybody else have also experienced such
behaviour.
Can it be because of OCT2014-SP1 release ??
Thanks
On Sun, Jan 18, 2015 at 11:20 AM, Martin Kersten
Hi
Each MAL instruction is executed by a single thread. Some operators require the complete input to be available, such as in your case the group.subgroup().
martin
On 18/01/15 01:27, shamsul hassan wrote:
Hi All,
I have recently install monetdb "OCT 2014-SP1" release on my Oracle Enterprise 6 Linux server.
Server configuration -
8 CPU Cores 64 GB RAM
I have loaded a data set with 70 columns having 15 Millions Rows.
I have started mserver5 as below --
mserver5 --debug=10 --set *gdk_nr_threads=8* --dbpath=/data/monetdb/testdb --set mapi_open=true --set mapi_port=54321 --set merovingian_uri=mapi:monetdb://localhost:54321/testdb --set mapi_usock=/data/monetdb/testdb/.mapi.sock --set monet_vault_key=/data/monetdb/testdb/.vaultkey --set sql_optimizer=default_pipe --set max_clients=64
Now I have ran a query as below --
select product_type,COUNT(DISTINCT sales_document),sum(ord_qty) FROM bookings where ord_entry_date between '2014-04-01' and '2014-10-01' group by product_type;
This query took around 90 Secs .. So I started tomograph and checked plotted graph ... which shows the parallelism usage as 39.9 % only and its utilizing only 1 thread with maximum time went in the step - group.subgroupdone almost 98%.
My Question is that even I have started the server with gdk_nr_threads=8 , then why it is just using single thread.
More info --
mserver5 --version
MonetDB 5 server v11.19.7 "Oct2014-SP1" (64-bit, 64-bit oids)
Copyright (c) 1993-July 2008 CWI
Copyright (c) August 2008-2014 MonetDB B.V., all rights reserved
Visit http://www.monetdb.org/ for further information
Found 63.0GiB available memory, 8 available cpu cores
Libraries:
libpcre: 7.8 2008-09-05 (compiled with 7.8)
openssl: OpenSSL 1.0.1e 11 Feb 2013 (compiled with OpenSSL 1.0.1e-fips 11 Feb 2013)
libxml2: 2.7.6 (compiled with 2.7.6)
Compiled by: root@localhost (x86_64-unknown-linux-gnu)
Compilation: gcc -O3 -fomit-frame-pointer -pipe -D_FORTIFY_SOURCE=2
Linking : /usr/bin/ld -m elf_x86_64
Any pointers ??
Thanks
_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list