Anyone has any tips please? This is halting our whole development. Please help! Thanks!
625GB available. Way bigger than the data set. So disk space no problemOn Mon, Mar 16, 2009 at 9:56 AM, Yue Sheng <yuesheng8@gmail.com> wrote:
could it be due to sql config settings? need bigger log file size perhaps? just guessing now...On Sat, Mar 14, 2009 at 8:56 PM, Yue Sheng <yuesheng8@gmail.com> wrote:
I don't have any indexing on the table.
On Thu, Mar 12, 2009 at 1:24 AM, Stefan Manegold <Stefan.Manegold@cwi.nl> wrote:
How exactly do you load your data (via RJDBC)?
A single
COPY 600000000 RECORDS INTO <table> FROM <file> USING DELIMITERS <...>;
or a single INSERT statement per row?
Does your table have constraints (keys, NOT NULL, ...) defined?
Stefan
| Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl |
On Thu, Mar 12, 2009 at 01:07:00AM -0700, Yue Sheng wrote:
> I have a new 1TB on the Machine, it has more than 800GB free; much more than
> what's needed... So that cannot be the problem.
>
> On Thu, Mar 12, 2009 at 12:26 AM, Martin Kersten <Martin.Kersten@cwi.nl>wrote:
>
> > Yue Sheng wrote:
> > > I'm been trying to insert from file for over 322 files, totally 12GB
> > > with over 600 million rows. The RJDBC connection hangs after 350 million
> > > row. I noticed the VSIZE in Activity Monitor is 30.85GB, closer to 32GB
> > > system memory. Is this the reason that I cannot connect to the database?
> > > What should I do?
> > >
> > > Error message I get from MonetDB server:
> > >
> > > # MonetDB server v5.8.4, based on kernel v1.26.4
> > > # Serving database 'bigdata', using 8 threads
> > > # Compiled for i386-apple-darwin9.6.0/64bit with 64bit OIDs dynamically
> > > linked
> > > # Copyright (c) 1993-July 2008 CWI.
> > > # Copyright (c) August 2008- MonetDB B.V., all rights reserved
> > > # Visit http://monetdb.cwi.nl/ for further information
> > > #warning: please don't forget to set your vault key!
> > > #(see /Users/someuser/MonetDB64/etc/monetdb5.conf)
> > > # Listening for connection requests on mapi:monetdb://127.0.0.1:50000/
> > > <http://127.0.0.1:50000/>
> > > # MonetDB/SQL module v2.26.4 loaded
> > > >!ERROR: GDKsave: failed name=24/2460, ext=tail, mode 0
> > > !ERROR: GDKsave: failed name=23/2317, ext=tail, mode 0
> > > !ERROR: GDKsave: failed name=25/2503, ext=tail, mode 0
> > > !ERROR: GDKsave: failed name=26/2657, ext=tail, mode 0
> > > !ERROR: GDKsave: failed name=22/2274, ext=tail, mode 0
> > > !ERROR: GDKsave: failed name=22/2224, ext=tail, mode 0
> > > !ERROR: GDKsave: failed name=01/112, ext=head, mode 0
> > > !ERROR: logger_exit: could not open
> > > /Users/someuser/MonetDB64/var/MonetDB5/sql_logs/bigdata/sql/log
> > This looks as if the device on which you attempt to write
> > is full.
> > >
> > >
> > >
> > > System:
> > > MacPro Dual Quad Xenon
> > > Leopard OS X
> > > 32GB RAM
> > > R 2.8.1 32Bit
> > > MonetDB v5.8.4 complied for i386-apple-darmin9.6.0/64bit
> > >
> > >
> > > Many thanks.
> > >
> > > Yuesheng
> > >
> > >
> > > ------------------------------------------------------------------------
> > >
> > >
> > ------------------------------------------------------------------------------
> > > Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are
> > > powering Web 2.0 with engaging, cross-platform capabilities. Quickly and
> > > easily build your RIAs with Flex Builder, the Eclipse(TM)based
> > development
> > > software that enables intelligent coding and step-through debugging.
> > > Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
> > >
> > >
> > > ------------------------------------------------------------------------
> > >
> > > _______________________________________________
> > > MonetDB-users mailing list
> > > MonetDB-users@lists.sourceforge.net
> > > https://lists.sourceforge.net/lists/listinfo/monetdb-users
> >
> >
> >
> > ------------------------------------------------------------------------------
> > Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are
> > powering Web 2.0 with engaging, cross-platform capabilities. Quickly and
> > easily build your RIAs with Flex Builder, the Eclipse(TM)based development
> > software that enables intelligent coding and step-through debugging.
> > Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
> > _______________________________________________
> > MonetDB-users mailing list
> > MonetDB-users@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/monetdb-users
> >
> ------------------------------------------------------------------------------
> Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are
> powering Web 2.0 with engaging, cross-platform capabilities. Quickly and
> easily build your RIAs with Flex Builder, the Eclipse(TM)based development
> software that enables intelligent coding and step-through debugging.
> Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
> _______________________________________________
> MonetDB-users mailing list
> MonetDB-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/monetdb-users
--
| CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ |
| 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 |
| The Netherlands | Fax : +31 (20) 592-4312 |
------------------------------------------------------------------------------
Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are
powering Web 2.0 with engaging, cross-platform capabilities. Quickly and
easily build your RIAs with Flex Builder, the Eclipse(TM)based development
software that enables intelligent coding and step-through debugging.
Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
_______________________________________________
MonetDB-users mailing list
MonetDB-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/monetdb-users