- users-list - monetdb.org

TypeException:user.main[2]:'A0' may not be used before being initialized
by Ruben Silva 08 Nov '13

08 Nov '13

Dear MonetDB users, Have you experienced anything like this before? TypeException:user.main[2]:'A0' may not be used before being initialized. Stacktrace follows: java.sql.SQLException: TypeException:user.main[2]:'A0' may not be used before being initialized at nl.cwi.monetdb.jdbc.MonetConnection$ResponseList.executeQuery(MonetConnection.java:2535) at nl.cwi.monetdb.jdbc.MonetConnection$ResponseList.processQuery(MonetConnection.java:2284) at nl.cwi.monetdb.jdbc.MonetStatement.internalExecute(MonetStatement.java:508) at nl.cwi.monetdb.jdbc.MonetStatement.execute(MonetStatement.java:349) at nl.cwi.monetdb.jdbc.MonetPreparedStatement.<init>(MonetPreparedStatement.java:118) at nl.cwi.monetdb.jdbc.MonetConnection.prepareStatement(MonetConnection.java:901) at nl.cwi.monetdb.jdbc.MonetConnection.prepareStatement(MonetConnection.java:825) at org.apache.commons.dbcp.DelegatingConnection.prepareStatement(DelegatingConnection.java:281) at org.apache.commons.dbcp.PoolingDataSource$PoolGuardConnectionWrapper.prepareStatement(PoolingDataSource.java:313) (...) What do you recommend? Should I open a bug report? Ruben Silva

2 3

Monetdb ODBC datatype conversion problem
by baraa Mohamad 06 Nov '13

06 Nov '13

Dear MonetDB users, I have created a database link from oracle to monetdb using Monetdb ODBC driver, the link works fine but i have a strange results 😓 I have the following configurations: host1: Oracle (ubuntu 64bit)host2: MonetDB (ubntu 64bit)ODBC driver: MonetDB odbc (64bit) using unixODBC driver manager on monetdb i have the following table: emp (id integer , name varchar); test query: select id from emp where id<20; when I execute this query directly on montdb I get the following results (IDs from 1 to 19)ID123..19 BUT when I execute this query from oracle using dblink select "id" from "emp"@monetdb where "id"<20 ; I GET THE FOLLOWING RESULTS: ID-------- 18589934592 21.2885E+10 31.7180E+10 42.1475E+10 52.5770E+10 63.0065E+10 73.4360E+10 83.8655E+10 94.2950E+10 10 I cannot figure out the cause of the problem and how I can solve it (it seems a problem related to datatype conversion using ODBC) Could you please tell me what I should do.... Best regards,Baraa

3 6

Modify column datataype
by baraa Mohamad 06 Nov '13

06 Nov '13

Hello, I would like to know if it is possible to change a column type. I have a table with millions of records and I wand to change the type of the column (id) but I couldn't find how to do that. Thanks in advance for your help, Regards,Baraa

3 3

Database corruption
by Pedro Salgueiro 04 Nov '13

04 Nov '13

Hi, We are using MonetDB combined with Mondrian for a while ago, and ever since we started put more load on it, with several users using the same instance, we are experiencing some serious data corruption with no reason, leading lost data of some tables and some times loosing the entire database. This weekend we had such an issue, for no apparent reason, the data got corrupted and apparently all data of a table got lost. We have no way of seeing what was being done before the data got corrupted, but since we were not loading data into it, the only queries that could be running were queries generated from Mondrian. In the logs, the only thing that we can find related to this issue is the following: 2013-11-04 11:17:56 MSG merovingian[22802]: database 'normal' (16384) was > killed by signal SIGSEGV which, results in a "connection terminated" After alot of debugging, we found out that issuing a simple query as "select count(*) from table" would cause this issue: sql>select count(*) FROM table; > Connection terminated Under the sql_logs in the database directory, there is a file named "log" with the following contents: 052001 9516 and a file named "log.9516" which is empty. When I do "\d table", I get the following result. It seems the table lost all its columns: sql>\d "table" > CREATE TABLE "sys"."table" ( > ); It's also important to note that this only happened only some tables. The rest of tables are ok. Don't know if it's important, the entire database uses about 390MB, so it's quite small. This seems to be quite a big issue leading to data loss, and worst, corrupted databases preventing its usage, unless the entire database is rebuilt. I have some questions regarding this issue which I never saw them answered, and might be very useful to any users of MonetDB: In a situation like this, is there any way to understand what happened to MonetDB in order to avoid such situation in the future? What are the "sql_logs" for? I see there are some files there, but can't find any meaning of the contents. Could these files be used to recover lost data? In previous mails to the mailing list Martin stated that the system logs all entries added to each table. Where are these logs located? Also, could they be used to recover lost data? If you want, we can provide the entire database so you can debug and try to understand what happened in order to prevent such situations in the future. Best regards, Pedro Salgueiro

1 0

Login Help
by Nitin Gautam 04 Nov '13

04 Nov '13

Hi I need to exectue some batch jobs on Monet which are supposed to run as cron jobs. Since the process runs in background I was hoping to use mclient to execute the queries. But when I use mclient I am prompted to enter a password. I could not see any option to specify the password too as an argument to mclient. Is there something I am missing or is there another way to do what I am trying to do i.e. not use mclient. Thanks Nitin

2 2

Char limitation for ruby-monetdb-sql
by Christian Kleinke 01 Nov '13

01 Nov '13

Hi, I have problem regarding executing a script with ruby-monetdb-sql. The connection works and simple statements are executed as expected. However, when I try to send a large script, nearly 20000 characters, nothing happens. Neither an exception is thrown nor a SQL error is produced by MonetDB. However, I use "\<" with the same 20000 character script, it works as expected. Two questions: 1. Is there a char limitation for ruby-monetdb-sql? 2. Is there a MonetDB-SQL command to load scripts, something like sql>load("my_script.sql")? Thank you very much in advance. Christian

2 1

MonetDB using all availaible disk space
by Ruben Silva 01 Nov '13

01 Nov '13

Hello, I'm performing some bulk inserts into some tables (COPY INTO). Meanwhile some JOINs between those tables are executed and also some DELETEs. During this JOIN's, the disk begins to fill up and only stops after there is no more space available (60GB in my case). Same for RAM. After a (long) while the space gets freed up. The COPY INTO's are done without any significant delay. The files that use that space are some .tail and .head in the database bat folder. If the queries are not concurrent there aren't any problems. The JOIN it's not using any PK or FK. Have some one noticed this before? Should I open a bug report? Ruben Silva * *

2 2

Using GeoSpatial functionality
by Dennis Pallett 30 Oct '13

30 Oct '13

Hello, I want to start using the GeoSpatial features of MonetDB and according to http://www.monetdb.org/Documentation/Extensions/GIS I need to rebuild MonetDB with the --enable-geom argument to do so. Does this mean that all the existing binaries published do not include the GeoSpatial functionality? If so, I need to follow the guide at http://www.monetdb.org/Developers/SourceCompile to build my own version of MonetDB, correct? Best regards, Dennis Pallett

2 2

Re: Are joins really this expensive?
by Bryan Senseman 30 Oct '13

30 Oct '13

Forwarding to user-list as this seems to have gotten lost Actually, I'm running a few different versions of MonetDB on different environments. July2012SP2, Feb2013SP4 & SP5. July2012SP2 on Windows and Linux (Ubuntu); Feb2013SP5 on Windows and Linux, and Feb2013SP4 on Linux. All were installed using pre-built packages/installers. The trace files themselves are from a Linux Feb2013SP4 installation. The Feb2013SP5 install was an inplace install on top of July2012SP2 and wasn't sure if different "data" versions would cloud the trace. To Martins questions (which I never got in my inbox...or junk) yes, date_hour_id is a unique key for the dim_date_hour table, but obviously duplicated within the fact_utilization_hourly table. My question was more generalized, but if a specific solution can be found, fantastic. In general I am seeing joins drastically reduce performance overall. All are running on EC2; m1.xlarge with High I/O Thanx, Bryan On Mon, Oct 28, 2013 at 9:10 AM, Bryan Senseman <monetdb(a)openbi.com> wrote: > Forwarding to user-list as this seems to have gotten lost > > Actually, I'm running a few different versions of MonetDB on different > environments. July2012SP2, Feb2013SP4 & SP5. July2012SP2 on Windows and > Linux (Ubuntu); Feb2013SP5 on Windows and Linux, and Feb2013SP4 on Linux. > All were installed using pre-built packages/installers. The trace files > themselves are from a Linux Feb2013SP4 installation. The Feb2013SP5 > install was an inplace install on top of July2012SP2 and wasn't sure if > different "data" versions would cloud the trace. > > To Martins questions (which I never got in my inbox...or junk) yes, > date_hour_id is a unique key for the dim_date_hour table, but obviously > duplicated within the fact_utilization_hourly table. > > My question was more generalized, but if a specific solution can be found, > fantastic. In general I am seeing joins drastically reduce performance > overall. > > All are running on EC2; m1.xlarge with High I/O > > Thanx, > Bryan > > ---------- Forwarded message ---------- > From: Bryan Senseman <monetdb(a)openbi.com> > Date: Sat, Oct 19, 2013 at 8:51 PM > Subject: Re: Are joins really this expensive? > To: Stefan Manegold <Stefan.Manegold(a)cwi.nl> > > > Ask, and ye shall receive, let me know if you need anything else; and I > can't share the data, nor would you want a many GB data bomb. > > Bryan > > > On Sat, Oct 19, 2013 at 3:17 PM, Stefan Manegold <Stefan.Manegold(a)cwi.nl>wrote: > >> Dear Bryan, >> >> unless you could share your data such that we could reproduce the problem, >> could you please profile both queries using TRACE, stethoscope or >> tomograph; >> cf., >> http://www.monetdb.org/Documentation/Cookbooks/SQLrecipes/QueryTiming >> and share the respective output? >> >> Then we might be able analyze what exactly causes the performance >> difference between both queries. >> >> Thanks, >> Stefan >> >> >> Bryan Senseman <monetdb(a)openbi.com> wrote: >>> >>> I have created a traditional star-schema in MonetDB, using both Linux >>> and Windows. We are using Mondrian OLAP on top so it generates the >>> standard OLAP-y type of query using group-bys of dimensional values. What >>> I’m finding is that doing joins in MonetDB is extremely expensive, for >>> example: >>> >>> >>> >>> The following query against a fact table with 261 million rows returns >>> in 9-11seconds consistently >>> >>> select date_hour_id/10000 as date_month_id, avg(seconds) from >>> fact_utilization_hourly group by date_month_id; >>> >>> returning 41 rows, the query includes a “Trick” to get to a month_year >>> value because the time_ids are smart keys… yyyyMMddhh. >>> >>> >>> >>> With the following query, using standard OLAP-y syntax, the query takes >>> over 5 mins consistently. >>> >>> select year_month_value, avg(seconds) from fact_utilization_hourly f >>> join dim_date_hour d on f.date_hour_id = d.date_hour_id group by >>> year_month_value; >>> >>> returning the same 41 rows. >>> >>> >>> >>> I’m at a loss to figure out why this join causes a 3200% percent >>> increase in response time. And this seems to have gotten worse in SP5 from >>> anecdotal evidence. >>> >>> >>> >>> Bryan >>> >>> ------------------------------ >>> >>> This transmission is confidential and intended solely for the use of the >>> recipient named above. It may contain confidential, proprietary, or legally >>> privileged information. If you are not the intended recipient, you are >>> hereby notified that any unauthorized review, use, disclosure or >>> distribution is strictly prohibited. If you have received this transmission >>> in error, please contact the sender by reply e-mail and delete the original >>> transmission and all copies from your system. >>> >>> ------------------------------ >>> >>> users-list mailing list >>> users-list(a)monetdb.org >>> >>> https://www.monetdb.org/mailman/listinfo/users-list >>> >>> >> -- >> | Stefan.Manegold(a)CWI.nl | Database Architectures (DA) | >> | www.CWI.nl/~manegold | Science Park 123 (L321) | >> | +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) | >> > > > -- ------------------------------ This transmission is confidential and intended solely for the use of the recipient named above. It may contain confidential, proprietary, or legally privileged information. If you are not the intended recipient, you are hereby notified that any unauthorized review, use, disclosure or distribution is strictly prohibited. If you have received this transmission in error, please contact the sender by reply e-mail and delete the original transmission and all copies from your system.

1 0

Dump database with other than admin-user - access denied
by Klaus Kruse 30 Oct '13

30 Oct '13

Hello, we will run MonetDB on two servers and while installing and so on went fine, I struggle with the backup solution. For this, we normally add a separate user with full rights. The dump seems to fail on the sys.db_user_info : mclient --format=sql --user=backup > /dump-file MAPI = backup@localhost:50000 QUERY = SELECT "name" FROM "sys"."auths" WHERE "name" NOT IN (SELECT "name" FROM "sys"."db_user_info") AND "grantor" <> 0 ORDER BY "name" ERROR = !SELECT: access denied for backup to table 'sys.db_user_info' Ok, let's give it a try: GRANT ALL ON sys.db_user_info TO "backup" ; operation successful (1.874ms) MAPI = backup@localhost:50000 QUERY = SELECT "ui"."name", "ui"."fullname", "password_hash"("ui"."name"), "s"."name" FROM "sys"."db_user_info" "ui", "sys"."schemas" "s" WHERE "ui"."default_schema" = "s"."id" AND "ui"."name" <> 'monetdb' ORDER BY "ui"."name" ERROR = !access denied for user 'backup' So, my question is: How to give another user than 'monetdb' full rights to dump the whole db? Thanks in advance und best regards Klaus Kruse -- Corpex Internet GmbH * Schauenburgerstrasse 6 * D-20095 Hamburg Tel: +49 40 822268-0 * Fax: +49 40 822268-100 * http://www.corpex.de/ HRB 78752, Amtsgericht Hamburg, Sitz: Hamburg, USt.-Id: DE 213001412 Geschäftsführer: Daniel Hanelt

1 0