On 2008-10-13 22:23, Richard Wesley wrote:
Hi All -
I am trying to evaluate MonetDB on Windows by loading up some large data sets. I'm not having much luck and I'm hoping someone can help me out.
The first set of problems revolve around the ODBC driver. It seems to not be able to report errors or even return the results of queries reliably. Most errors come back as a generic error message although we occasionally run into strange messages involving UTF-16 conversions when no string columns are involved (e.g. a count distinct of a numeric column). We are using the July 2008 version from the web site. Are these known issues and is there a more recent build we could try? This is not in production yet, we are simply trying to evaluate the system.
I have managed to work around the ODBC problems very slowly and have got to a point where I can load small data sets. But when I then try to do a large insert using the COPY FROM syntax, my client got a "connection terminated" error. When I take the COPY FROM statement and paste it into the client, the MonetDB server crashes. The only log files I can find have a few (mostly unprintable) characters in them. Restarting the server and trying again produces the same problem. The insert query is
COPY 6568253 RECORDS INTO "Airlines"."_2005_On_Time_Performance" FROM 'C:\\DOCUME~1\\hawkfish\\LOCALS~1\\Temp\\1\\7A.tmp\\table.tbl' USING DELIMITERS '\t', '\n', '\'';
Is there some limit on the COPY INTO syntax? Is there some log file somewhere that could point me in the right direction?
The log you found is probably the write-ahead log of the database server which is used to guarantee transactions. The ODBC driver can produce a log if you start the process that loads the driver with an environment variable ODBCDEBUG with as value the file you would like to use to collect the log (i.e. something like set ODBCDEBUG=c:\odbc.log ) What kind of system are you using? 32 bit or 64 bit? XP? Vista? If a 32 bit system, remember that all data has to be addressable (i.e. less than 2GB), and this includes all temporary tables that are used while loading. (The database can be bigger, but all data that is referred to (used) in a query has to be addressable at the same time.) Conversions happen because the interface to the server uses UTF-8 and often the interface from the application to the driver (actually, the interface between the driver manager and the driver) uses wide characters (i.e. UTF-16). But I would be interested in fixing any bugs in that area, so if you can capture a log session where it goes wrong, I'd be grateful. -- Sjoerd Mullender