Re: users-list Digest, Vol 29, Issue 14

20 Jan 2015


      On 20/01/15 08:23, bhuvan Mitra wrote:
...
Thank you stefan.
Is there any way through which I can get to know about those implementations or buying it from actian is the only way?
Actually I want to contribute to c-store, open source column oriented database in this compression/decompression area. This answer from you will be very worthy to me.
Then you better get up to date with the status of that project and its successors in industry ;)
http://db.csail.mit.edu/projects/cstore/
Martin
Thanks in advance.
On Mon, Jan 19, 2015 at 9:29 PM, mailto:users-list-request@monetdb.org> wrote:
Send users-list mailing list submissions to
    users-list@monetdb.org mailto:users-list@monetdb.org
To subscribe or unsubscribe via the World Wide Web, visit
    https://www.monetdb.org/mailman/listinfo/users-list
    or, via email, send a message with subject or body 'help' to
    users-list-request@monetdb.org mailto:users-list-request@monetdb.org
You can reach the person managing the list at
    users-list-owner@monetdb.org mailto:users-list-owner@monetdb.org
When replying, please edit your Subject line so it is more specific
    than "Re: Contents of users-list digest..."
Today's Topics:
1. Re: users-list Digest, Vol 29, Issue 12 (Stefan Manegold)
        2. Next release? (g4@novadsp.com mailto:g4@novadsp.com)
        3. Re: Not utilising Mutithreads (shamsul hassan)
        4. Re: Next release? (Sjoerd Mullender)
----------------------------------------------------------------------
Message: 1
    Date: Mon, 19 Jan 2015 14:58:13 +0100 (CET)
    From: Stefan Manegold mailto:Stefan.Manegold@cwi.nl>
    To: Communication channel for MonetDB users mailto:users-list@monetdb.org>
    Subject: Re: users-list Digest, Vol 29, Issue 12
    Message-ID: <895297092.253122.1421675893625.JavaMail.zimbra@cwi.nl mailto:895297092.253122.1421675893625.JavaMail.zimbra@cwi.nl>
    Content-Type: text/plain; charset=utf-8
Hi,
MonetDB and Actian/VectorWise are two different systems that do not share any code.
MonetDB does not use any compression in the way Actian/VectorWise does.
Best,
    Stefan
----- Original Message -----
     > Thank you Martin.
     > So, did monetdb bought the codes from actian code base and included in it's
     > source code?
     > Also kindly tell me if monetdb is following any other compression techniques
     > apart from pfor, pfor-delta and pdict.
     > Thanks in advance.
     >
     > On Mon, Jan 19, 2015 at 4:30 PM, < users-list-request@monetdb.org mailto:users-list-request@monetdb.org > wrote:
     >
     >
     > Send users-list mailing list submissions to
     > users-list@monetdb.org mailto:users-list@monetdb.org
     >
     > To subscribe or unsubscribe via the World Wide Web, visit
     > https://www.monetdb.org/mailman/listinfo/users-list
     > or, via email, send a message with subject or body 'help' to
     > users-list-request@monetdb.org mailto:users-list-request@monetdb.org
     >
     > You can reach the person managing the list at
     > users-list-owner@monetdb.org mailto:users-list-owner@monetdb.org
     >
     > When replying, please edit your Subject line so it is more specific
     > than "Re: Contents of users-list digest..."
     >
     >
     > Today's Topics:
     >
     > 1. Re: Not utilising Mutithreads (Martin Kersten)
     > 2. regarding source code (bhuvan Mitra)
     > 3. Re: regarding source code (Martin Kersten)
     > 4. information about compression/decompression techniques of
     > monetdb (bhuvan Mitra)
     >
     >
     > ----------------------------------------------------------------------
     >
     > Message: 1
     > Date: Sun, 18 Jan 2015 12:20:15 +0100
     > From: Martin Kersten < martin@monetdb.org mailto:martin@monetdb.org >
     > To: users-list@monetdb.org mailto:users-list@monetdb.org
     > Subject: Re: Not utilising Mutithreads
     > Message-ID: < 54BB96EF.3030906@monetdb.org mailto:54BB96EF.3030906@monetdb.org >
     > Content-Type: text/plain; charset=windows-1252; format=flowed
     >
     > Hi
     >
     > Each MAL instruction is executed by a single thread.
     > Some operators require the complete input to be available,
     > such as in your case the group.subgroup().
     >
     > martin
     >
     > On 18/01/15 01:27, shamsul hassan wrote:
     > > Hi All,
     > >
     > > I have recently install monetdb "OCT 2014-SP1" release on my Oracle
     > > Enterprise 6 Linux server.
     > >
     > > Server configuration -
     > >
     > > 8 CPU Cores
     > > 64 GB RAM
     > >
     > > I have loaded a data set with 70 columns having 15 Millions Rows.
     > >
     > > I have started mserver5 as below --
     > >
     > > mserver5 --debug=10 --set *gdk_nr_threads=8* --dbpath=/data/monetdb/testdb
     > > --set mapi_open=true --set mapi_port=54321 --set
     > > merovingian_uri=mapi:monetdb://localhost:54321/testdb --set
     > > mapi_usock=/data/monetdb/testdb/.mapi.sock --set
     > > monet_vault_key=/data/monetdb/testdb/.vaultkey --set
     > > sql_optimizer=default_pipe --set max_clients=64
     > >
     > > Now I have ran a query as below --
     > >
     > > select product_type,COUNT(DISTINCT sales_document),sum(ord_qty) FROM
     > > bookings where ord_entry_date between '2014-04-01' and '2014-10-01' group
     > > by product_type;
     > >
     > > This query took around 90 Secs ..
     > > So I started tomograph and checked plotted graph ... which shows the
     > > parallelism usage as 39.9 % only and its utilizing only 1 thread with
     > > maximum time went in the step - group.subgroupdone almost 98%.
     > >
     > > My Question is that even I have started the server with gdk_nr_threads=8 ,
     > > then why it is just using single thread.
     > >
     > > More info --
     > >
     > > mserver5 --version
     > >
     > > MonetDB 5 server v11.19.7 "Oct2014-SP1" (64-bit, 64-bit oids)
     > >
     > > Copyright (c) 1993-July 2008 CWI
     > >
     > > Copyright (c) August 2008-2014 MonetDB B.V., all rights reserved
     > >
     > > Visit http://www.monetdb.org/ for further information
     > >
     > > Found 63.0GiB available memory, 8 available cpu cores
     > >
     > > Libraries:
     > >
     > > libpcre: 7.8 2008-09-05 (compiled with 7.8)
     > >
     > > openssl: OpenSSL 1.0.1e 11 Feb 2013 (compiled with OpenSSL 1.0.1e-fips 11
     > > Feb 2013)
     > >
     > > libxml2: 2.7.6 (compiled with 2.7.6)
     > >
     > > Compiled by: root@localhost (x86_64-unknown-linux-gnu)
     > >
     > > Compilation: gcc -O3 -fomit-frame-pointer -pipe -D_FORTIFY_SOURCE=2
     > >
     > > Linking : /usr/bin/ld -m elf_x86_64
     > >
     > > Any pointers ??
     > >
     > > Thanks
     > >
     > >
     > >
     > > _______________________________________________
     > > users-list mailing list
     > > users-list@monetdb.org mailto:users-list@monetdb.org
     > > https://www.monetdb.org/mailman/listinfo/users-list
     > >
     >
     >
     >
     > ------------------------------
     >
     > Message: 2
     > Date: Mon, 19 Jan 2015 12:28:05 +0530
     > From: bhuvan Mitra < bhuvanrjb@gmail.com mailto:bhuvanrjb@gmail.com >
     > To: users-list@monetdb.org mailto:users-list@monetdb.org
     > Subject: regarding source code
     > Message-ID:
     > < CAMmryP6eg5AAMY1x70KJZMdc3k7P8aqjnHuS5xN-fNSJ8mN_sQ@mail.gmail.com mailto:CAMmryP6eg5AAMY1x70KJZMdc3k7P8aqjnHuS5xN-fNSJ8mN_sQ@mail.gmail.com >
     > Content-Type: text/plain; charset="utf-8"
     >
     > Hello,
     >
     > I am very interested to know and learn about implementations of
     > compression/decompression techniques that monetdb is following. I have seen
     > the monetdb's light weight compression algorithms(pfor, pfor-delta, pdict),
     > cache involved decompression techniques and excited to know how actually
     > are they implemented in form of codes. Kindly tell me in which folder/file
     > is the source code of this compression/decompression algorithms present. I
     > have downloaded the source tarball from the website and could not find
     > these code snippets. Plese help me to identify the path.
     >
     > Regards,
     > mg
     >