Anyone has any tips please? This is halting our whole development. Please help! Thanks!

On Mon, Mar 16, 2009 at 9:58 AM, Yue Sheng <yuesheng8@gmail.com> wrote:
625GB available. Way bigger than the data set. So disk space no problem


On Mon, Mar 16, 2009 at 9:56 AM, Yue Sheng <yuesheng8@gmail.com> wrote:
could it be due to sql config settings? need bigger log file size perhaps? just guessing now...


On Sat, Mar 14, 2009 at 8:56 PM, Yue Sheng <yuesheng8@gmail.com> wrote:
I don't have any indexing on the table. 


On Thu, Mar 12, 2009 at 1:24 AM, Stefan Manegold <Stefan.Manegold@cwi.nl> wrote:

How exactly do you load your data (via RJDBC)?
A single
       COPY 600000000 RECORDS INTO <table> FROM <file> USING DELIMITERS <...>;
or a single INSERT statement per row?

Does your table have constraints (keys, NOT NULL, ...) defined?

Stefan


On Thu, Mar 12, 2009 at 01:07:00AM -0700, Yue Sheng wrote:
> I have a new 1TB on the Machine, it has more than 800GB free; much more than
> what's needed... So that cannot be the problem.
>
> On Thu, Mar 12, 2009 at 12:26 AM, Martin Kersten <Martin.Kersten@cwi.nl>wrote:
>
> > Yue Sheng wrote:
> > > I'm been trying to insert from file for over 322 files, totally 12GB
> > > with over 600 million rows. The RJDBC connection hangs after 350 million
> > > row. I noticed the VSIZE in Activity Monitor is 30.85GB, closer to 32GB
> > > system memory. Is this the reason that I cannot connect to the database?
> > > What should I do?
> > >
> > > Error message I get from MonetDB server:
> > >
> > > # MonetDB server v5.8.4, based on kernel v1.26.4
> > > # Serving database 'bigdata', using 8 threads
> > > # Compiled for i386-apple-darwin9.6.0/64bit with 64bit OIDs dynamically
> > > linked
> > > # Copyright (c) 1993-July 2008 CWI.
> > > # Copyright (c) August 2008- MonetDB B.V., all rights reserved
> > > # Visit http://monetdb.cwi.nl/ for further information
> > > #warning: please don't forget to set your vault key!
> > > #(see /Users/someuser/MonetDB64/etc/monetdb5.conf)
> > > # Listening for connection requests on mapi:monetdb://127.0.0.1:50000/
> > > <http://127.0.0.1:50000/>
> > > # MonetDB/SQL module v2.26.4 loaded
> > >  >!ERROR: GDKsave: failed name=24/2460, ext=tail, mode 0
> > > !ERROR: GDKsave: failed name=23/2317, ext=tail, mode 0
> > > !ERROR: GDKsave: failed name=25/2503, ext=tail, mode 0
> > > !ERROR: GDKsave: failed name=26/2657, ext=tail, mode 0
> > > !ERROR: GDKsave: failed name=22/2274, ext=tail, mode 0
> > > !ERROR: GDKsave: failed name=22/2224, ext=tail, mode 0
> > > !ERROR: GDKsave: failed name=01/112, ext=head, mode 0
> > > !ERROR: logger_exit: could not open
> > > /Users/someuser/MonetDB64/var/MonetDB5/sql_logs/bigdata/sql/log
> > This looks as if the device on which you attempt to write
> > is full.
> > >
> > >
> > >
> > > System:
> > > MacPro Dual Quad Xenon
> > > Leopard OS X
> > > 32GB RAM
> > > R 2.8.1 32Bit
> > > MonetDB v5.8.4 complied for i386-apple-darmin9.6.0/64bit
> > >
> > >
> > > Many thanks.
> > >
> > > Yuesheng
> > >
> > >
> > > ------------------------------------------------------------------------
> > >
> > >
> > ------------------------------------------------------------------------------
> > > Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are
> > > powering Web 2.0 with engaging, cross-platform capabilities. Quickly and
> > > easily build your RIAs with Flex Builder, the Eclipse(TM)based
> > development
> > > software that enables intelligent coding and step-through debugging.
> > > Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
> > >
> > >
> > > ------------------------------------------------------------------------
> > >
> > > _______________________________________________
> > > MonetDB-users mailing list
> > > MonetDB-users@lists.sourceforge.net
> > > https://lists.sourceforge.net/lists/listinfo/monetdb-users
> >
> >
> >
> > ------------------------------------------------------------------------------
> > Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are
> > powering Web 2.0 with engaging, cross-platform capabilities. Quickly and
> > easily build your RIAs with Flex Builder, the Eclipse(TM)based development
> > software that enables intelligent coding and step-through debugging.
> > Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
> > _______________________________________________
> > MonetDB-users mailing list
> > MonetDB-users@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/monetdb-users
> >

> ------------------------------------------------------------------------------
> Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are
> powering Web 2.0 with engaging, cross-platform capabilities. Quickly and
> easily build your RIAs with Flex Builder, the Eclipse(TM)based development
> software that enables intelligent coding and step-through debugging.
> Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
> _______________________________________________
> MonetDB-users mailing list
> MonetDB-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/monetdb-users


--
| Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl |
| CWI,  P.O.Box 94079 | http://www.cwi.nl/~manegold/  |
| 1090 GB Amsterdam   | Tel.: +31 (20) 592-4212       |
| The Netherlands     | Fax : +31 (20) 592-4312       |

------------------------------------------------------------------------------
Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are
powering Web 2.0 with engaging, cross-platform capabilities. Quickly and
easily build your RIAs with Flex Builder, the Eclipse(TM)based development
software that enables intelligent coding and step-through debugging.
Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
_______________________________________________
MonetDB-users mailing list
MonetDB-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/monetdb-users