Re: [MonetDB-users] How to set the memory limit for monetDB.

30 Apr 2008

      In fact, I've wrapped monetDB using JENA's api. And tried to load DBPedia's
RDF data with it. It runs OK on small dataset. But it always requires memory
during the insertion. So it failed when there's no memory to be allocated.
Since it has been  wrapped by JENA, I know few of the background schema of
the database.
I go with you it may caused by the os's vm strategy. I'll check it.
I've seen that there are some debug option for the config file. Can I use
some of them to log something that can let you know my problem? Which one
should I use ?

On Wed, Apr 30, 2008 at 2:19 PM, Martin Kersten 
wrote:
...
Kang Zhang wrote:
...
The data's size is about 3 Gigabytes.  21,852,641 triples need to be
inserted in total.
My machine is 64-bit and only 8G memory for that.
Can you say a little more when and how it crashed.
Did you use the SQL COPY command?
Is it a victim process selected by the OS, due to excessive virtual
memory use?
Do you have an gdb stack trace where it happen?
Thank you for your immediate reply.
Kang.
On Wed, Apr 30, 2008 at 1:21 PM, Martin Kersten mailto:Martin.Kersten@cwi.nl> wrote:
Kang Zhang wrote:
    > I am using the latest version of monetDB from the web-site(the
super
    > source tarball).
    > My machine is a X86-64 machine with a red-hat installed on it.
    When I
    > insert data into monetDB,
    > it always requires memory from the system. Then it will be crashed
    > with running out of memory.
    > Is there a way to set the limit for memory used by monetDB ?
    I've read
    > the docs on the web site
    > and can't get what I need from that.
    Dear Kang,
    indeed, there is no way to limit the memory use. All columns are
    memory
    mapped files,
    which give an upperbound on their size. This is particularly
    problematic
    for 32-bits machines and very large columns.
    >
    > Thank you.
    >
    > --
    > ------------------------------------------------------------
    > Kang Zhang
    >
    > Computer Science Dept.
    > Shanghai Jiao Tong University
    >
------------------------------------------------------------------------
...
>
    >
-------------------------------------------------------------------------
...
> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
    > Don't miss this year's exciting event. There's still time to
    save $100.
    > Use priority code J8TL2D2.
    >
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javao...
...
>
------------------------------------------------------------------------
...
>
    > _______________________________________________
    > MonetDB-users mailing list
    > MonetDB-users@lists.sourceforge.net
    mailto:MonetDB-users@lists.sourceforge.net
    > https://lists.sourceforge.net/lists/listinfo/monetdb-users
    >
-------------------------------------------------------------------------
...
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
    Don't miss this year's exciting event. There's still time to save
    $100.
    Use priority code J8TL2D2.
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javao...
...
_______________________________________________
    MonetDB-users mailing list
    MonetDB-users@lists.sourceforge.net
    mailto:MonetDB-users@lists.sourceforge.net
    https://lists.sourceforge.net/lists/listinfo/monetdb-users
--
------------------------------------------------------------
Kang Zhang
Computer Science Dept.
Shanghai Jiao Tong University
------------------------------------------------------------------------
-------------------------------------------------------------------------
...
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
Don't miss this year's exciting event. There's still time to save $100.
Use priority code J8TL2D2.
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javao...
...
------------------------------------------------------------------------
_______________________________________________
MonetDB-users mailing list
MonetDB-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/monetdb-users
-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
Don't miss this year's exciting event. There's still time to save $100.
Use priority code J8TL2D2.
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javao...
_______________________________________________
MonetDB-users mailing list
MonetDB-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/monetdb-users
-- 
------------------------------------------------------------
Kang Zhang

Computer Science Dept.
Shanghai Jiao Tong University