That's great, would you mind illustrating how to deal with (1) having
mixture of text and .gz files in the same directory
(2) dealing with error (mostly the rare encounter of "non-standard" columns)
both (1) & (2) are currently taken care of in R (for (2), i just ignore the
error message and press on), but I'm not sure how to deal with them in shell
script. hints also welcome. Just need pointers on where to start.
thanks.
On Mon, Mar 16, 2009 at 5:25 PM, Stefan Manegold
On Mon, Mar 16, 2009 at 04:56:31PM -0700, Yue Sheng wrote:
no i/o, swapping, no activity in CPU monitor 32GB
haven't tried. manual loading >600 files then see when it break is an option, albert a painful one...
assiming your files are in .../DIR/*.csv:
rows="`cat .../DIR/*.csv | wc -l`" for i in .../DIR/*.csv ; do if [ "$rows" ] ; then mclient -lsql -s"copy $ROWS records into usrtable from '$i' using delimiters '|';" rows='' else mclient -lsql -s"copy `cat $i | wc -l` records into usrtable from '$i' using delimiters '|';" fi done
Stefan
------------------------------------------------------------------------------
Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are powering Web 2.0 with engaging, cross-platform capabilities. Quickly and easily build your RIAs with Flex Builder, the Eclipse(TM)based development software that enables intelligent coding and step-through debugging. Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
-- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |
------------------------------------------------------------------------------ Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are powering Web 2.0 with engaging, cross-platform capabilities. Quickly and easily build your RIAs with Flex Builder, the Eclipse(TM)based development software that enables intelligent coding and step-through debugging. Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users