I could give an upper bound but cannot give absolute. Because I used error suppression in my R code such that it "skips" row that give insert error (such as missing column).

This suppression method seems to work well for all other inserts. So shouldn't be the cause of problem

Would it be possible to tell MonteDB to "pre-allocate" more memory then it currently does? the files have a upper bound of how much rows it will have.

Thanks.

On Mon, Mar 16, 2009 at 10:20 AM, Stefan Manegold <Stefan.Manegold@cwi.nl> wrote:
On Thu, Mar 12, 2009 at 09:15:17AM -0700, Yue Sheng wrote:
> This is the loop I used, as I'm inserting a file at a time.
>
>
> drv <- JDBC("nl.cwi.monetdb.jdbc.MonetDriver",
> "/Users/someuser/monetDB64/share/MonetDB/lib/monetdb-1.9-jdbc.jar")
>
> conn <- dbConnect(drv, "jdbc:monetdb://localhost/bigdata","username",
> "password")
>
>
> for (i in 1:length(filenames)) {
>
> str <- paste("copy into usrtable from '", wholename[i],"' using delimiters
> '|';", sep="")
>
> try(dbSendUpdate(conn, str))
>
> }
>

Could you try giving the first "copy into" a hint of how many records you
plan to insert in total, e.g.,

COPY <N> RECORDS INTO ...

where <N> is the sum of all records from all files you load,

(and optionally give each subsequent "copy into" likewise a hint of how many
records the respective file holds).

This could help MonetDB memory management ...

Stefan

>
> where filesnames are file names and wholename are same as filenames but has
> directory info as well.
>
>
> Thanks.
>
>
>
> On Thu, Mar 12, 2009 at 3:27 AM, Tom H <tom@limepepper.co.uk> wrote:
>
> > Yue Sheng wrote:
> > >     > !ERROR: logger_exit: could not open
> > >     > /Users/someuser/MonetDB64/var/MonetDB5/sql_logs/bigdata/sql/log
> > >     This looks as if the device on which you attempt to write
> > >     is full.
> >
> > How big is the /Users partition on the disk?
> > # df -h
> >
> > Tom
> >
> >
> > ------------------------------------------------------------------------------
> > Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are
> > powering Web 2.0 with engaging, cross-platform capabilities. Quickly and
> > easily build your RIAs with Flex Builder, the Eclipse(TM)based development
> > software that enables intelligent coding and step-through debugging.
> > Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
> > _______________________________________________
> > MonetDB-users mailing list
> > MonetDB-users@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/monetdb-users
> >

> ------------------------------------------------------------------------------
> Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are
> powering Web 2.0 with engaging, cross-platform capabilities. Quickly and
> easily build your RIAs with Flex Builder, the Eclipse(TM)based development
> software that enables intelligent coding and step-through debugging.
> Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
> _______________________________________________
> MonetDB-users mailing list
> MonetDB-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/monetdb-users


--
| Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl |
| CWI,  P.O.Box 94079 | http://www.cwi.nl/~manegold/  |
| 1090 GB Amsterdam   | Tel.: +31 (20) 592-4212       |
| The Netherlands     | Fax : +31 (20) 592-4312       |

------------------------------------------------------------------------------
Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are
powering Web 2.0 with engaging, cross-platform capabilities. Quickly and
easily build your RIAs with Flex Builder, the Eclipse(TM)based development
software that enables intelligent coding and step-through debugging.
Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
_______________________________________________
MonetDB-users mailing list
MonetDB-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/monetdb-users