November 2009 - users-list

Re: [MonetDB-users] COPY INTO memory issues
by Jens Oehlschlägel 25 Nov '09

25 Nov '09

> > This wasn't the fact. Again monetdb consumed all my memory and when loading > > finished it indicated that it was still doing something (CPU @ ~20%). > log records have to be written, your OS may kick in to flush dirty pages, > reloading swapped out processes, .... The following might not apply to Linux, but still: under windows, suddenly flushing dirty pages from large memory mapped files can severly degrade the system performance ("stalled"). I know windows has settings to limit the amount of dirty pages that may accumulate without flushing. Maybe Linux has something similar and maybe this helps here. Please keep me in the loop if you try this route. Jens

1 0

[MonetDB-users] 'create table as select...' from union queries doesn't work
by Alex Bo. 25 Nov '09

25 Nov '09

Hi, If I do this: create table t1 (f1 int); create table t2 (f1 int); create table t_union as (select f1 from t1 union all select f1 from t2) with data; Then no table is created and no exception is thrown. Should I open a bug for this? (AugSP2 on Windows) -- View this message in context: http://old.nabble.com/%27create-table-as-select...%27-from-union-queries-do… Sent from the monetdb-users mailing list archive at Nabble.com.

2 1

[MonetDB-users] create function
by Paul Phillips [ gmail ] 24 Nov '09

24 Nov '09

Thank you for the find piped help to the src documentation. I can make the following function using mclient from AUG2009SP2 well: create function f(i integer) returns bool begin return true; end; It seems that the create function is failing only in the SQL editor for SQuirreL client 3.0.2. I am using the compiled jdbc jar of MonetDB-AUG2009SP2 and have tried to use a compiled sourceball version of jar from 11NOV2009 current branch. both provide this error: Error: syntax error, unexpected $end, expecting WHILE in: "create function f(i integer) returns bool begin return true;" SQLState: null ErrorCode: 0 Error occured in: create function f(i integer) returns bool begin return true Should this then be a question for SQuirrel Client? Thanks Paul

1 0

[MonetDB-users] temporary tables
by Alex Bo. 24 Nov '09

24 Nov '09

Hi, How do temporary tables behave? Are they only stored in memory or are they written to the disk too like normal tables? What performance should I expect from using "create temporary table..." and then querying it in comparison to having a nested query? Thanks. -- View this message in context: http://old.nabble.com/temporary-tables-tp26504075p26504075.html Sent from the monetdb-users mailing list archive at Nabble.com.

2 1

[MonetDB-users] Database Failure
by Matthew Jones 24 Nov '09

24 Nov '09

Hey folks, had a database running for several weeks without any problems but early this morning anytime a client would try to connect to it the connection would just hang. After shutting down and restarting I see this: 2009-11-23 09:38:06 MSG merovingian[28036]: starting Merovingian 1.1 for dbfarm /opt/MonetDB5/dbfarm 2009-11-23 09:38:06 MSG merovingian[28036]: listening for TCP connections on 0.0.0.0:50000 2009-11-23 09:38:06 MSG discovery[28036]: listening for UDP messages on 0.0.0.0:50000 2009-11-23 09:38:06 MSG merovingian[28036]: handling commands over UNIX socket /opt/MonetDB5/dbfarm/.merovingian_control 2009-11-23 09:38:06 MSG discovery[28036]: new neighbour 10.1.0.79 2009-11-23 09:38:07 MSG discovery[28036]: new database mapi:monetdb:// hsdw01:50000/reporting_primary (ttl=660s) 2009-11-23 09:39:45 MSG merovingian[28036]: starting database 'reporting_primary' due to control signal 2009-11-23 09:39:45 MSG merovingian[28036]: database 'reporting_primary' has crashed after start on 2009-11-23 09:36:59, attempting restart, up min/avg/max: 2m/2d/1w, crash average: 1.00 0.60 0.27 (18-10=8) 2009-11-23 09:39:46 MSG reporting_primary[28067]: arguments: /usr/bin/mserver5 --config=/etc/monetdb5.conf --dbname=reporting_primary --dbinit=include sql; --set monet_daemon=yes --set mapi_open=false --set mapi_autosense=true --set mapi_port=50001 --set monet_vault_key=/opt/MonetDB5/dbfarm/reporting_primary /.vaultkey 2009-11-23 09:39:46 MSG reporting_primary[28067]: # MonetDB server v5.14.2, based on kernel v1.32.2 2009-11-23 09:39:46 MSG reporting_primary[28067]: # Serving database 'reporting_primary', using 8 threads 2009-11-23 09:39:46 MSG reporting_primary[28067]: # Compiled for x86_64-pc-linux-gnu/64bit with 64bit OIDs dynamically linked 2009-11-23 09:39:46 MSG reporting_primary[28067]: # Copyright (c) 1993-July 2008 CWI. 2009-11-23 09:39:46 MSG reporting_primary[28067]: # Copyright (c) August 2008-2009 MonetDB B.V., all rights reserved 2009-11-23 09:39:46 MSG reporting_primary[28067]: # Visit http://monetdb.cwi.nl / for further information 2009-11-23 09:39:46 MSG reporting_primary[28067]: # Listening for connection requests on mapi:monetdb://127.0.0.1:50001/ 2009-11-23 09:39:46 MSG reporting_primary[28067]: # MonetDB/SQL module v2.32.2 loaded 2009-11-23 09:39:46 ERR reporting_primary[28067]: !mvc_init: unable to create system tables 2009-11-23 09:39:55 ERR merovingian[28036]: failed to fork mserver: database 'reporting_primary' has inconsistent state (running but dead), review merovingian's logfile for any peculiarities The database marks itself as running but any attempt to connect to it returns the following error: monetdb ERROR MALException:setScenario:Scenario not initialized 'sql' I've read other problems reports that had a similar issue and the suggestion was to wait for the sql module to allow connections but even after 5 minutes this is all that is returned. Is there any way to get this database back to a consistent state so that it is usable? 73, Matthew W. Jones (KI4ZIB) http://matburt.net

8 20

[MonetDB-users] Mx documentation
by Paul Phillips [ gmail ] 24 Nov '09

24 Nov '09

Can someone tell where the Mx utility is for creating the documentation? I have seen references in sources to this command and searches are trouble with only two letters like mx. Is this documentation different/more than the nl website? Thank you. Paul

3 3

[MonetDB-users] limit resident memory usage
by Roberto Cornacchia 23 Nov '09

23 Nov '09

Hi, Is there a way to limit the usage of -resident- memory? Say I have 8GB RAM. What I would like is being able to say: no matter what, use up to 2GB resident memory, and as much virtual memory as you like. I have tried to play around with gdk_mem_maxsize and gkd_mmap_minsize, but with no luck so far. Maybe I haven't very clear yet how they work? Roberto

1 0

[MonetDB-users] strange UNION exception
by Alex Bo. 19 Nov '09

19 Nov '09

Hi, Every once in a while when using a UNION query I get an exception that looks like this: !UNION: column counts (3 and 2) do not match (the numbers '3' and '2' vary from case to case) The message isn't right, because the queries I run do have matching column counts between the Unions. This happens to me in various scenarios. The thing that is common to all the cases is that it seems to happen when some sort of threshold is passed. For example, I can have a working query with 50 OR clauses in the WHERE statement that works, but as soon as I add the 51th OR clause, I get the exception. In other cases I get the same exception with a different amount of OR clauses, but again I see the behavior of the exception appearing when adding one more OR clause to a working query. The thing that is in common to all the failing queries is that there's always a join on two tables and I get data from both tables with a lot of OR clauses. If necessary I can upload data and queries that illustrate this, but the queries are rather long. Please advise. Thanks. -- View this message in context: http://old.nabble.com/strange-UNION-exception-tp26421401p26421401.html Sent from the monetdb-users mailing list archive at Nabble.com.

2 2

[MonetDB-users] COPY with empty fields
by pphillips99＠gmail.com 19 Nov '09

19 Nov '09

COPY 1 OFFSET 100 RECORDS INTO tlbc FROM '/mnt/data/myfile.txt' DELIMITERS '\t', '\r\n' Trying to use COPY to import tab delimited text files. The text file has a header row and I've verified the delimiters and eols. I have a number of empty columns at the end of each row in the text file and those colums in the table are a combination of integers and decimals. Received the following error at the first of the empty columns: !SQLException:sql:value '' while parsing from line 0 field 213 not inserted, expecting type lng !SQLException:importTable:failed to import table !ERROR: Decimal cannot be parsed from lng I realize I could do manipulations but it's second choice given the data size, but the file is 8G. How can I overcome? Paul

4 7

[MonetDB-users] CSV file to a table
by Nafees Ur Rehman 17 Nov '09

17 Nov '09

Hi, I am trying to read a CSV file content to a table. But I think Im making mistake in the syntax somehow. Can any one please guide me a little? Regards Nafees Ur Rehman

3 3