Re: users-list Digest, Vol 29, Issue 14

Thank you stefan. Is there any way through which I can get to know about those implementations or buying it from actian is the only way? Actually I want to contribute to c-store, open source column oriented database in this compression/decompression area. This answer from you will be very worthy to me. Thanks in advance. On Mon, Jan 19, 2015 at 9:29 PM, wrote:

Send users-list mailing list submissions to users-list@monetdb.org

To subscribe or unsubscribe via the World Wide Web, visit https://www.monetdb.org/mailman/listinfo/users-list or, via email, send a message with subject or body 'help' to users-list-request@monetdb.org

You can reach the person managing the list at users-list-owner@monetdb.org

When replying, please edit your Subject line so it is more specific than "Re: Contents of users-list digest..."

Today's Topics:

1. Re: users-list Digest, Vol 29, Issue 12 (Stefan Manegold) 2. Next release? (g4@novadsp.com) 3. Re: Not utilising Mutithreads (shamsul hassan) 4. Re: Next release? (Sjoerd Mullender)

----------------------------------------------------------------------

Message: 1 Date: Mon, 19 Jan 2015 14:58:13 +0100 (CET) From: Stefan Manegold To: Communication channel for MonetDB users Subject: Re: users-list Digest, Vol 29, Issue 12 Message-ID: <895297092.253122.1421675893625.JavaMail.zimbra@cwi.nl> Content-Type: text/plain; charset=utf-8

Hi,

MonetDB and Actian/VectorWise are two different systems that do not share any code.

MonetDB does not use any compression in the way Actian/VectorWise does.

Best, Stefan

...

Thank you Martin. So, did monetdb bought the codes from actian code base and included in it's source code? Also kindly tell me if monetdb is following any other compression techniques apart from pfor, pfor-delta and pdict. Thanks in advance.

On Mon, Jan 19, 2015 at 4:30 PM, < users-list-request@monetdb.org > wrote:

Send users-list mailing list submissions to users-list@monetdb.org

To subscribe or unsubscribe via the World Wide Web, visit https://www.monetdb.org/mailman/listinfo/users-list or, via email, send a message with subject or body 'help' to users-list-request@monetdb.org

You can reach the person managing the list at users-list-owner@monetdb.org

When replying, please edit your Subject line so it is more specific than "Re: Contents of users-list digest..."

Today's Topics:

1. Re: Not utilising Mutithreads (Martin Kersten) 2. regarding source code (bhuvan Mitra) 3. Re: regarding source code (Martin Kersten) 4. information about compression/decompression techniques of monetdb (bhuvan Mitra)

----------------------------------------------------------------------

Message: 1 Date: Sun, 18 Jan 2015 12:20:15 +0100 From: Martin Kersten < martin@monetdb.org > To: users-list@monetdb.org Subject: Re: Not utilising Mutithreads Message-ID: < 54BB96EF.3030906@monetdb.org > Content-Type: text/plain; charset=windows-1252; format=flowed

Hi

Each MAL instruction is executed by a single thread. Some operators require the complete input to be available, such as in your case the group.subgroup().

martin

On 18/01/15 01:27, shamsul hassan wrote:

...

Hi All,

I have recently install monetdb "OCT 2014-SP1" release on my Oracle Enterprise 6 Linux server.

Server configuration -

8 CPU Cores 64 GB RAM

I have loaded a data set with 70 columns having 15 Millions Rows.

I have started mserver5 as below --

mserver5 --debug=10 --set *gdk_nr_threads=8* --dbpath=/data/monetdb/testdb --set mapi_open=true --set mapi_port=54321 --set merovingian_uri=mapi:monetdb://localhost:54321/testdb --set mapi_usock=/data/monetdb/testdb/.mapi.sock --set monet_vault_key=/data/monetdb/testdb/.vaultkey --set sql_optimizer=default_pipe --set max_clients=64

Now I have ran a query as below --

select product_type,COUNT(DISTINCT sales_document),sum(ord_qty) FROM bookings where ord_entry_date between '2014-04-01' and '2014-10-01' group by product_type;

This query took around 90 Secs .. So I started tomograph and checked plotted graph ... which shows the parallelism usage as 39.9 % only and its utilizing only 1 thread with maximum time went in the step - group.subgroupdone almost 98%.

My Question is that even I have started the server with gdk_nr_threads=8 , then why it is just using single thread.

More info --

mserver5 --version

MonetDB 5 server v11.19.7 "Oct2014-SP1" (64-bit, 64-bit oids)

Copyright (c) 1993-July 2008 CWI

Copyright (c) August 2008-2014 MonetDB B.V., all rights reserved

Visit http://www.monetdb.org/ for further information

Found 63.0GiB available memory, 8 available cpu cores

Libraries:

libpcre: 7.8 2008-09-05 (compiled with 7.8)

openssl: OpenSSL 1.0.1e 11 Feb 2013 (compiled with OpenSSL 1.0.1e-fips 11 Feb 2013)

libxml2: 2.7.6 (compiled with 2.7.6)

Compiled by: root@localhost (x86_64-unknown-linux-gnu)

Compilation: gcc -O3 -fomit-frame-pointer -pipe -D_FORTIFY_SOURCE=2

Linking : /usr/bin/ld -m elf_x86_64

Any pointers ??

Thanks

_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

------------------------------

Message: 2 Date: Mon, 19 Jan 2015 12:28:05 +0530 From: bhuvan Mitra < bhuvanrjb@gmail.com > To: users-list@monetdb.org Subject: regarding source code Message-ID: < CAMmryP6eg5AAMY1x70KJZMdc3k7P8aqjnHuS5xN-fNSJ8mN_sQ@mail.gmail.com > Content-Type: text/plain; charset="utf-8"

Hello,

I am very interested to know and learn about implementations of compression/decompression techniques that monetdb is following. I have seen the monetdb's light weight compression algorithms(pfor, pfor-delta,

----- Original Message ----- pdict),

...

cache involved decompression techniques and excited to know how actually are they implemented in form of codes. Kindly tell me in which folder/file is the source code of this compression/decompression algorithms present. I have downloaded the source tarball from the website and could not find these code snippets. Plese help me to identify the path.

Regards, mg