- developers-list - monetdb.org

Re: Issues with MonetDB
by Stefan Manegold 23 Jan '13

23 Jan '13

Dear Ashish, thank you very much for your feedback. Did you already try Oct2012-SP2, upcoming Oct2012-SP3 (currently available from http://dev.monetdb.org/downloads/testing/ ; will be formally released soon), or the Feb2013 branch from Mercurial (http://dev.monetdb.org/hg/MonetDB/branches)? To be able to further debug the problem on our side, we would need all the ingredients required to reproduce the problem you're experiencing (code, data, queries, detailed instructions). Best, Stefan Ashish Kumar Singh <ashishkumar.singh(a)altair.com> wrote: Dear Monet DB team, We are facing few very critical issues in monetdb in one of the reporting applications we are trying to put up using it. First of all performance improvements are really great using monetdb but we are facing some issues related to data consistency as described below: 1. Duplicate tuples are getting inserted few times. 2. Updates to database are not available for read immediately , sometimes wrong values are returned by database. After certain number of retires these becomes available though can go to 10 times of retires. We are facing this for Oct 2012 Sp1 build. These data related bug are not acceptable in our usage scenario and this will be the case with other folks too! Question to the list is there any released version backwards / upwards which can be used instead of October 2012 Sp1 which does not have these issues? I have filled a bug also for this at http://bugs.monetdb.org/show_bug.cgi?id=3222 Regards, Ashish

2 2

Announcement: New Oct2012-SP3 Bugfix release of MonetDB suite
by Sjoerd Mullender 22 Jan '13

22 Jan '13

The MonetDB team at CWI/MonetDB BV is pleased to announce the Oct2012-SP3 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <http://www.monetdb.org/>. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Oct 2012-SP3 bugfix release Bug Fixes * 2291: small doubles end up as NULL after arithmetic * 2579: SQL: incorrect cast from double to int * 2659: tuples INTERSECT vs count (... INTERSECT ...) differs * 2805: row_number doesn't work in complex query * 2886: warning: Catalog parameter '' ignored * 2977: timestamp minus date results in program error * 2978: now() + 1 results in illegal argument * 2999: Up-cast from SQL drops order information? * 3048: algebra.join undefined when using multiple arithmetic operations in SQL where-clause * 3124: insert into table with two foreign keys crashes mserver5 * 3136: Memleak (in querycache?) while querying * 3143: Type resolution error in SQL procedural code * 3163: mserver5 crashes with large number of concurrent clients * 3168: crash when insert varchar/char/clob with default NULL * 3169: php driver uses call-time pass-by-reference * 3173: SQL queries with limit and offset clauses are unable to use bind parameters for either * 3176: Same query and different results * 3181: Join query returns zero results when there is an index on the tables * 3185: inconsistency between documentation and implementation of command line options * 3186: case evaluted to early * 3188: Storage of client history * 3189: Overflow in calculation * 3190: When subtracting two columns of type "timestamp", no results are returned * 3191: rel_optimizer.c:5596: _rel_optimizer: Assertion `0' failed. * 3192: "No such column with index" error if CASE in SELECT clause * 3199: Inconspicuous sequence of prepare, execute, commit and close produces error * 3205: Inet datatype handling inconsistent between int, cast( as inet) and string * 3208: Unable to execute prepared statements with a large number of arguments * 3209: exp_bin: Assertion `0' failed * 3212: Query cache does not work properly

1 0

Issues with MonetDB
by Ashish Kumar Singh 22 Jan '13

22 Jan '13

Dear Monet DB team, We are facing few very critical issues in monetdb in one of the reporting applications we are trying to put up using it. First of all performance improvements are really great using monetdb but we are facing some issues related to data consistency as described below: 1. Duplicate tuples are getting inserted few times. 2. Updates to database are not available for read immediately , sometimes wrong values are returned by database. After certain number of retires these becomes available though can go to 10 times of retires. We are facing this for Oct 2012 Sp1 build. These data related bug are not acceptable in our usage scenario and this will be the case with other folks too! Question to the list is there any released version backwards / upwards which can be used instead of October 2012 Sp1 which does not have these issues? I have filled a bug also for this at http://bugs.monetdb.org/show_bug.cgi?id=3222 Regards, Ashish

2 1

Fw: Inconsistency in data selection. count*
by Tapomay Dey 21 Jan '13

21 Jan '13

Forwarding to dev forum. Not appropriate for user list. ----- Forwarded Message ----- From: Tapomay Dey <tapomay(a)yahoo.com> To: Communication channel for Monet DB users <users-list(a)monetdb.org> Sent: Monday, January 21, 2013 2:39 PM Subject: Inconsistency in data selection. count* select count(*) from table; Results: 168466 rows select count(*) from table where column=0; Results: 168466 rows select count(*) from table where column<>0; Results: 168466 rows When I do a "select column from table where client_id=0 limit 5;" I see correct non-zero values in the result although the selection condition is that the column value must be 0. I put the data on another instance/machine - I don't see this issue. Plz find a screenshot@ https://picasaweb.google.com/115470548136322675830/January212013?authuser=0… Another case: sql>select count(*) from table2 where column2=0; | 391326 | sql>select count(*) from table2 where column2<>0; | 141108 | sql>select count(*) from table2; | 399326 | sql>select column2 from table2 where column2=0 limit 5; +-----------+ | column2 | +===========+ | 53 | | 53 | | 53 | | 53 | | 53 | +-----------+ 5 tuples (156.631ms) Regards, Tapomay _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

3 2

Error in merovingian log while performing a specific select: gdk_batop.c insert_string_bat: Assertion failed
by Tapomay Dey 18 Jan '13

18 Jan '13

ERR msearch_stats_db[4515]: mserver5: gdk_batop.c:175: insert_string_bat: Assertion `v >= ((var_t) (((1<<10) * sizeof(stridx_t)) >> 0))' failed. I see above error in merovingian.log while performing a specific select query and the client says "Connection terminated": select id, col2, col3, col4, col5, timestamp, count(*) from stats_daily where id=38 group by "id", "col2", "col3", "col4", "timestamp", "col5" having count(*) >1; There's a unique key on the table stats_daily on (id,col2,col3, col4, timestamp, col5). However following query succeeds(it just has more selection conditions): select id, col2, col3, col4, col5, timestamp, count(*) from stats_daily where col2 <> '' and col3 <> '' and col4 <> '' and col5 <> '' and id <> 0 group by "id", "col2", "col3", "col4", "timestamp", "col5" having count(*) >1; One thing I know for sure is that there was an error in inserting records into the DB at some point before: Following is an excerpt from the merovingian logs: 2013-01-10 10:40:49 ERR msearch_stats_db[19210]: !mal_mapi.listen: expected filedescriptor, but received something else 2013-01-10 10:40:57 ERR merovingian[8996]: client error: client [XX.XX.XX.XX]:51573 sent challenge in incomplete block: 2013-01-10 10:45:29 ERR msearch_stats_db[19210]: mserver5: bat_storage.c:40: delta_bind_del: Assertion `b' failed. 2013-01-10 10:45:29 ERR msearch_stats_db[19210]: mserver5: bat_storage.c:40: delta_bind_del: Assertion `b' failed. 2013-01-10 23:48:18 ERR msearch_stats_db[31637]: !mal_mapi.listen: expected filedescriptor, but received something else 2013-01-10 23:48:19 ERR msearch_stats_db[31637]: !mal_mapi.listen: expected filedescriptor, but received something else 2013-01-10 23:48:32 ERR msearch_stats_db[31637]: !FATAL: BBPdir: subcommit attempted without backup BBP.dir. 2013-01-11 04:29:31 ERR msearch_stats_db[4480]: mserver5: gdk_batop.c:175: insert_string_bat: Assertion `v >= ((var_t) (((1<<10) * sizeof(stridx_t)) >> 0))' failed. Also, I keep getting "!mal_mapi.listen: expected filedescriptor, but received something else" almost every second(Performing continuous inserts into the db). Also attaching a subset of mervingian log that may contain other kinds of ERR s than above. What other diagnostics can I do? PS: Running on Feb2013 branch. changeset: 38061:d1a99a110b63 Ubuntu 12.10 64 bit. Regards, Tapomay. _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

2 3

MonetDB Clustering
by nguewo dominique 15 Jan '13

15 Jan '13

Dear Developers, I have a database in two systems with the same name. They both have a table with the same name. I have set the shared properties as follows. laptop1% monetdb -h laptop2-P mypassphrase set shared=pump/1/ pump laptop1% monetdb -h laptop3-P mypassphrase set shared=pump/2/ pump From a master node om trying to query the database to access data from the two tables but its not working. How t go about it? _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

2 1

Strange behaviour with MonetDB
by Ashish Kumar Singh 15 Jan '13

15 Jan '13

Hi List , We are facing one strange behavior with Monetdb. We have a simple table called test with following structure: CREATE SEQUENCE "testtable_seq" as bigint START WITH 1; CREATE TABLE testtable (dim_id bigint DEFAULT NEXT VALUE FOR "testtable_seq", dim_type varchar(100) NOT NULL, dim_value varchar(100) NOT NULL, CONSTRAINT unique_dimension UNIQUE(dim_type,dim_value) , CONSTRAINT pk_testtable PRIMARY KEY (dim_id)); We have a program which reads and writes to this table along with other tables using JDBC. We are facing below problems interminently: 1. When we query back data from this table with query like : select dim_id,dim_type,dim_value from testtable where dim_type='user' and dim_value='user1' we are getting back ID for some other value present in testable . 2. We are also seeing duplicate values getting inserted in to this table. Please suggest , we are using Oct SP1 build. Regards, Ashish _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

1 0

undefined symbol: msab_init while starting monetdb
by Tapomay Dey 14 Jan '13

14 Jan '13

I keep getting the following error while trying to start monetdb. msearch_stats_db[20642]: /usr/bin/mserver5: symbol lookup error: /usr/bin/mserver5: undefined symbol: msab_init FYI inititally I was getting !FATAL: GDKlockHome: could not create /usr/local/var/monetdb5/dbfarm/demo which I solved by changing the permissions of the specified folder following which I keep getting the above error. Regards, Tapomay. _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

2 2

Bulk Load, Same Table, Sequential COPY INTO, MAPI
by Brandon Jackson 13 Jan '13

13 Jan '13

Dear developers, In the last two weeks we have made a lot of progress on the MonetDB Streaming Bulk Loader Step in the Pentaho Data Integration. Yesterday, I tried to put it into use similar to a larger scale production use and started getting some errors along the lines of: "COMMIT: transaction is aborted because of concurency conflicts, will ROLLBACK instead" Not sure when these clear up. In this case after previous errors like this, I established a new connection and wanted to simply create a table. sql>CREATE TABLE "testing"."fact_cen_hist" ( "CENSUS_SK" INT , "FACILITY_SK" INT , "RESIDENT_SK" INT , "RESIDENT_LOCATION_SK" INT , "DISCHARGED_TO" CLOB , "PAYOR" VARCHAR(20) , "PAYOR_CATEGORY" VARCHAR(20) , "CENSUS_STATUS" VARCHAR(50) , "CENSUS_EVENT" VARCHAR(50) , "IS_ACTIVE" INT , "IS_MEDICAID_DAY" INT , "IS_MEDICARE_DAY" INT , "IS_HOSPICE_DAY" INT , "IS_COMMERCIAL_INSURANCE_DAY" INT , "IS_PRIVATE_PAY_DAY" INT , "IS_MANAGED_CARE_DAY" INT , "IS_MEDICARE_SECOND_PAYOR" INT , "IS_MEDICAID_PENDING_DAY" INT , "DATE_OF_STAY" INT , "NEXT_EVENT_DATE" INT , "DATE_BEFORE_NEXT_EVENT" INT , "CENSUS_EVENT_DATE" INT , "IS_MEMORY_CARE_DAY" INT , "IS_AL_DAY" INT , "IS_SNF_DAY" INT , "IS_DISCHARGED" INT , "IS_RTA" INT , "ROOM_KEY" VARCHAR(5) , "IS_BED_HOLD" INT , "BILLABLE_DAY" INT , "HEAD_IN_BED_DAY" INT , "FINANCIAL_DAY" INT , "IS_ADMISSION" INT , "IS_READMISSION" INT , "IS_EXPIRED" INT , "PAYOR_SK" INT , "IS_SUBMISSION" INT , "SPLIT_CENSUS" INT ) ; operation successful (2.7s) COMMIT: transaction is aborted because of concurency conflicts, will ROLLBACK instead sql> ---- Generally speaking, our bulk loader reads in a chunk of rows into memory say 10k, 20k or even 100k or 1 Million rows, then it will generate a COPY INTO, know how many rows are outbound; stream the rows through a BufferedWriter with all the new lines, and reading from BufferedReader and waiting for prompts etc. Then PDI will fetch another chunk of rows and repeat the process. I noticed that when using the MAPI API and trying to append a COMMIT in the output brought back an error saying that COMMIT is not allowed in auto commit mode, so it looks like after each big COPY INTO statement sends its complete pack of rows, with the new line stuff, should immediately try to COMMIT. What I want to figure out is why after the first COPY INTO of however many rows, 10k for example, would I get a message about failure to commit because of concurrency, ROLLBACK instead. Once that happens, is MonetDB in a funk? Do I need to shut it down and restart? What can I do to provide more helpful information to understand the nature of the problem? I read on: http://www.monetdb.org/Documentation/Cookbooks/SQLrecipies/LoadingBulkData That I could supply a guess much larger on the first COPY INTO so MonetDB can prepare itself for a huge influx of rows, despite getting a smaller batch. Example in my mind is know that my table has roughly 6 million rows, so let 6 million be the guess, then have really accurate calls where COPY INTO is told 20k rows are on the way in each subsequent call leading up to the 6 million that I guessed would be necessary. It worries me that this appears to be brittle. We might produce 10 million rows in a source table and want to stream it over. At current, it looks like COMMIT concurrency errors would come up as we sequentially do COPY INTO's building up to finally having the bulk load complete. Any ideas or feedback is very much appreciated. I can carefully provide code, more detail and follow any procedures necessary to help diagnose the issue. Sample data is also available. Thanks ahead of time, Brandon Jackson _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

1 0

Solution: jdbcclient.jar is always prompting even there is password list in .monetdb
by sigit kartiko 10 Jan '13

10 Jan '13

I use jdbcclient.jar to backup several table periodically with my crontab. I already put .monetdb with my password text there. But there is problem, it' s always prompting for password. I check the source and i make it right with this solution: before : if (user != equals(copts.getOption("user").getArgument()) pass = null; after: if (!user.equals(copts.getOption("user").getArgument())) pass = null; regards _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

3 4