January 2013 - users-list

Email archives for the forums
by Tapomay Dey 18 Jan '13

18 Jan '13

I don't see any forum archives since you switched from sourceforge. It's helpful to refer to other email threads. Is it there somewhere? I see the older one here: http://mail.monetdb.org/pipermail/developers-list/ and till 8th Jan '13 on sourceforge. Regards, Tapomay. _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

3 2

Monetdb on Solaris/Sparc: high system time problem
by Javier Picorel 17 Jan '13

17 Jan '13

Dear all, I'm running MonetDB (Dec2011-SP2) on Sparc Solaris 10 u9. I'm testing the TPC-H queries using SF10 and SF100 as the scale factors. My machine has 4 cores and 32GB of RAM and my farm is on a disk array. Unfortunately, there are some queries (e.g., Qry 7, 8, 9) that enters into phase in which the User/System time is 4% to 21% for a couple of minutes. Interestingly, the same issue doesn't show up on a Linux machine with a similar setup and the queries' system time is pretty low (<5%). Interestingly, vmstat output doesn't show any disk activity during the phase where the process is just in system, only there are high values on "re" and "mf" fields. There is no swapping going on ("po" and "pi" values are 0 in vmstat). These are the parameters of the DB: type default database shared default yes nthreads default 4 optpipe default_pipe master default no slave default <unknown> readonly default no nclients default 64 The way that I test the query is by running just one at a time. How can I mitigate this OS time? Thanks in advance, Javier _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

2 1

MonetDB Cross Partition Joins
by Wolf Rödiger 17 Jan '13

17 Jan '13

Hello, According to its man page MonetDB is able to work as a cluster. However, I found no documentation on cross partition joins. Is there a way to partition a relation in MonetDB across servers and perform a cross partition join? Regards Wolf _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

2 1

JDBC error: 'protocol violation: unexpected line in data block: .'
by Luis Neves 11 Jan '13

11 Jan '13

Hello all, Is this a known bug? Is there any workaround? * Stack Trace java.sql.SQLException: 'protocol violation: unexpected line in data block: .' at nl.cwi.monetdb.jdbc.MonetConnection$ResponseList.executeQuery(MonetConnection.java:2535) at nl.cwi.monetdb.jdbc.MonetConnection$ResponseList.processQuery(MonetConnection.java:2284) at nl.cwi.monetdb.jdbc.MonetStatement.internalExecute(MonetStatement.java:508) at nl.cwi.monetdb.jdbc.MonetStatement.execute(MonetStatement.java:349) at nl.cwi.monetdb.jdbc.MonetPreparedStatement.execute(MonetPreparedStatement.java:247) at nl.cwi.monetdb.jdbc.MonetPreparedStatement.executeQuery(MonetPreparedStatement.java:265) * Table definition sql>\d sys.wa_blogs_audience CREATE TABLE "sys"."wa_blogs_audience" ( "tid" CHAR(32) NOT NULL, "ts" BIGINT, "ts_hour" BIGINT, "ts_day" BIGINT, "ts_month" BIGINT, "s_site_name" VARCHAR(32), "s_hostname" VARCHAR(256), "n_pageviews" DOUBLE, "n_visits" DOUBLE, "n_visitors" DOUBLE, "h_visitors_register" VARCHAR(1881), CONSTRAINT "wa_blogs_audience_tid_pkey" PRIMARY KEY ("tid") ); * SQL SELECT ts, hllagg(h_visitors_register) FROM wa_blogs_audience WHERE ts>=1355172393000 AND ts<=1357764393000 GROUP BY ts; * Output from mclient $ mclient -p9701 kpi_v1 -f raw -s "SELECT ts, hllagg(h_visitors_register) FROM wa_blogs_audience WHERE ts>=1355172393000 AND ts<=1357764393000 GROUP BY ts;" at: <https://dl.dropbox.com/u/279023/misc/mclient_output.txt> * JDBC debug log at: <https://dl.dropbox.com/u/279023/misc/monet_1357773164928.log> Thanks in advance! -- Luis Neves _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

2 6

Perf. benchmark for monetdb
by nitin sharma 10 Jan '13

10 Jan '13

Hi there, I am evaluating monetdb and was wondering if there are performance bench mark numbers like the one published in C-store paper: http://people.csail.mit.edu/tdanford/6830papers/stonebraker-cstore.pdf I am looking for perf. number for 100 millions rows in fact table (500 GB) joined with couple of dimension tables. Does moentdb supports horizontal partitioning of data and joins across machine (distributed joins) ? Thanks!Nitin _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

2 3

Weird concurrency issue with JDBC getConnection
by Percy Wegmann 10 Jan '13

10 Jan '13

Dear Monet Team, I've been using the JDBC driver and have recently noticed that under load, calls to DriverManager.getConnection() will sometimes appear to hang. I was able to resolve this problem by serializing access to getConnection() using a synchronized block. I haven't had a chance to dig through the JDBC driver code to see what would account for this, but I figured I'd make you aware just in case someone else runs into this. Cheers, -- Percy Wegmann +1 512 637 8500 ext 148 _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

3 4

What is normally done to recover from 'crash'ed databases.
by Brandon Jackson 08 Jan '13

08 Jan '13

Hello, I have a toy database on my laptop that I've been using while testing development of a Pentaho Data Integration plugin for bulk loading. One day I shutdown my machine without stopping monetdb. When I turned on the computer and started monetdbd on the database everything seems to work fine. 'monetdb status' outputs the following: brandon-jacksons-macbook-pro:bin bjackson$ ./monetdb status name state uptime health demo crashed 0%, 0s crash 2013-01-05 00:29:20 brandon-jacksons-macbook-pro:bin bjackson$ ./monetdb start demo starting database 'demo'... done brandon-jacksons-macbook-pro:bin bjackson$ ./monetdb status name state uptime health demo running 5s 50%, 0s crash 2013-01-05 00:29:20 brandon-jacksons-macbook-pro:bin bjackson$ I cannot really tell if there is any real damage. What do people normally do to check the integrity of their databases and repair damage if it exists? Does a set of actions restore the health of the database? If I missed some documentation please feel free to refer me to some links. Thanks for the information. Brandon _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

3 3

Fwd: Re: c3p0 ConnectionTester socket read stuck after SQLException
by Hannes Mühleisen 08 Jan '13

08 Jan '13

Forwarding this to the list, please keep the discussion there... -------- Original Message -------- Subject: Re: c3p0 ConnectionTester socket read stuck after SQLException Date: Tue, 8 Jan 2013 08:17:35 -0800 (PST) From: Tapomay Dey <tapomay(a)yahoo.com> Reply-To: Tapomay Dey <tapomay(a)yahoo.com> To: Hannes.Muehleisen(a)cwi.nl <Hannes.Muehleisen(a)cwi.nl> My operations are as follows: 1. Create connections with auto-commit=false 2. Read queries from an external module. 3. Execute queries on the connection acquired in step 1 4. Update external module status as per result of queries. 5. If all is well, commit the connection. 6. If commit fails(optimistic locking is the culprit here), revert changes to the external module. All of these steps are implemented in a command chain pattern code. Thus I need to be able to pass around the connection. Thanks and Regards, Tapomay. ------------------------------------------------------------------------ *From:* Hannes Mühleisen <Hannes.Muehleisen(a)cwi.nl> *To:* Communication channel for MonetDB users <users-list(a)monetdb.org> *Sent:* Thursday, January 3, 2013 4:19 PM *Subject:* Re: c3p0 ConnectionTester socket read stuck after SQLException Hello Tapomay, after looking at your code, I was wondering why you chose the double management of connections in your Main/JdbcMonetdbTestClient classes. C3P0 provides a way to define "connection customizers" (com.mchange.v2.c3p0.ComboPooledDataSource.setConnectionCustomizerClassName(String)), which you can use to disable autocommit on new connections. This should simplify your example considerably. Nevertheless, I will try to provoke the exceptions you have mentioned and see how the JDBC driver handles them together with C3P0. Best, Hannes On 20/12/12 15:41, Tapomay Dey wrote: > As the source code has many unrelated dependencies I tried to extract > the relevant code. > Please find attached an eclipse project for the same. > However I haven't been able to reproduce the issue in this test project. > It should give you a fair idea of the sequence of calls made. Plz note > that a single insert query used in the live project is much much larger > (10K approx.) than the one in the test. > I reconfirmed that the issue is still there in the larger project. > > PS: I had sent this code long back and it got moderated just now(due to > some 40KB limit). Thus I have removed the dependencies > monetdb-jdbc-2.6.jar and c3p0-0.9.1.2.jar from lib. Plz add them to lib > folder and everything should work(after u set the correct DB names in > the code).(Anyways this code doesn't reproduce the issue I am facing so > I guess just a manual overview should suffice) > > Regards, > Tapomay. > > ------------------------------------------------------------------------ > *From:* Hannes Mühleisen <Hannes.Muehleisen(a)cwi.nl <mailto:Hannes.Muehleisen@cwi.nl>> > *To:* Communication channel for MonetDB users <users-list(a)monetdb.org <mailto:users-list@monetdb.org>> > *Sent:* Thursday, December 20, 2012 2:17 PM > *Subject:* Re: c3p0 ConnectionTester socket read stuck after SQLException > > Hello Tapomay, > > I will have a look later today, could you please also provide the source > code that you have used to provoke the issue? > > Thanks, > > Hannes > > On 20/12/12 08:26, Tapomay Dey wrote: > > Please find the thread dump below. > > Running MonetDB Database Server v1.6 (Oct2012-SP1). Compiled > > from MonetDB-11.13.5 src distribution. > > C3P0 version: 0.9.1.2. Also tried with 0.9.2. > > Occurs only after SQLException sometimes: INSERT INTO xyz : table xyz > > doesn't exist. > > Otherwise it works great. > > Also tried setting c3p0's preferredQuery for ConnectionTester to "select > > 1;". Problem persists. > > > > Name: pool-15-thread-2 > > State: RUNNABLE > > Total blocked: 0 Total waited: 1 > > > > Stack trace: > > java.net.SocketInputStream.socketRead0(Native Method) > > java.net.SocketInputStream.read(SocketInputStream.java:150) > > java.net.SocketInputStream.read(SocketInputStream.java:121) > > java.io.BufferedInputStream.fill(BufferedInputStream.java:235) > > java.io.BufferedInputStream.read1(BufferedInputStream.java:275) > > java.io.BufferedInputStream.read(BufferedInputStream.java:334) > > - locked java.io.BufferedInputStream@5cec5796 > > > nl.cwi.monetdb.mcl.net.MapiSocket$BlockInputStream._read(MapiSocket.java:846) > > > nl.cwi.monetdb.mcl.net.MapiSocket$BlockInputStream.readBlock(MapiSocket.java:895) > > > nl.cwi.monetdb.mcl.net.MapiSocket$BlockInputStream.read(MapiSocket.java:965) > > sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:283) > > sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:325) > > sun.nio.cs.StreamDecoder.read(StreamDecoder.java:177) > > - locked java.io.InputStreamReader@206a6883 > > java.io.InputStreamReader.read(InputStreamReader.java:184) > > java.io.BufferedReader.fill(BufferedReader.java:154) > > java.io.BufferedReader.readLine(BufferedReader.java:317) > > - locked java.io.InputStreamReader@206a6883 > > java.io.BufferedReader.readLine(BufferedReader.java:382) > > > nl.cwi.monetdb.mcl.io.BufferedMCLReader.readLine(BufferedMCLReader.java:119) > > > nl.cwi.monetdb.mcl.io.BufferedMCLReader.waitForPrompt(BufferedMCLReader.java:196) > > - locked nl.cwi.monetdb.mcl.io.BufferedMCLReader@29f38115 > > > nl.cwi.monetdb.jdbc.MonetConnection$ResponseList.executeQuery(MonetConnection.java:2296) > > - locked nl.cwi.monetdb.mcl.net.MapiSocket@319f9f37 > > > nl.cwi.monetdb.jdbc.MonetConnection$ResponseList.processQuery(MonetConnection.java:2274) > > > nl.cwi.monetdb.jdbc.MonetStatement.internalExecute(MonetStatement.java:508) > > nl.cwi.monetdb.jdbc.MonetStatement.execute(MonetStatement.java:349) > > nl.cwi.monetdb.jdbc.MonetStatement.executeQuery(MonetStatement.java:525) > > > com.mchange.v2.c3p0.impl.DefaultConnectionTester.activeCheckConnection(DefaultConnectionTester.java:73) > > > com.mchange.v2.c3p0.impl.DefaultConnectionTester.statusOnException(DefaultConnectionTester.java:143) > > > com.mchange.v2.c3p0.AbstractConnectionTester.statusOnException(AbstractConnectionTester.java:82) > > > com.mchange.v2.c3p0.impl.NewPooledConnection.handleThrowable(NewPooledConnection.java:437) > > - locked com.mchange.v2.c3p0.impl.NewPooledConnection@3340302c > > > com.mchange.v2.c3p0.impl.NewProxyStatement.executeUpdate(NewProxyStatement.java:78) > > > com.sokrati.moneta.mclient.JdbcMonetdbClient.executeUpdate(JdbcMonetdbClient.java:154) > > > com.sokrati.moneta.multiplexer.MonetdbClientCallable.call(MonetdbClientCallable.java:46) > > > com.sokrati.moneta.multiplexer.MonetdbClientCallable.call(MonetdbClientCallable.java:1) > > java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > > java.util.concurrent.FutureTask.run(FutureTask.java:166) > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > > java.lang.Thread.run(Thread.java:722) > > > > Regards, > > Tapomay. > > > > > > _______________________________________________ > > users-list mailing list > > users-list(a)monetdb.org <mailto:users-list@monetdb.org> <mailto:users-list@monetdb.org <mailto:users-list@monetdb.org>> > > http://mail.monetdb.org/mailman/listinfo/users-list > > > > _______________________________________________ > users-list mailing list > users-list(a)monetdb.org <mailto:users-list@monetdb.org> <mailto:users-list@monetdb.org <mailto:users-list@monetdb.org>> > http://mail.monetdb.org/mailman/listinfo/users-list > > > > > _______________________________________________ > users-list mailing list > users-list(a)monetdb.org <mailto:users-list@monetdb.org> > http://mail.monetdb.org/mailman/listinfo/users-list > _______________________________________________ users-list mailing list users-list(a)monetdb.org <mailto:users-list@monetdb.org> http://mail.monetdb.org/mailman/listinfo/users-list _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

1 0

the dbname option disappeared
by Stefan de Konink 04 Jan '13

04 Jan '13

I have justed checked out the Feb2013 release and noticed that the dbname option is gone, is there a substitute? Stefan _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

5 8

c3p0 ConnectionTester socket read stuck after SQLException
by Tapomay Dey 03 Jan '13

03 Jan '13

Please find the thread dump below. Running MonetDB Database Server v1.6 (Oct2012-SP1). Compiled from MonetDB-11.13.5 src distribution. C3P0 version: 0.9.1.2. Also tried with 0.9.2. Occurs only after SQLException sometimes: INSERT INTO xyz : table xyz doesn't exist. Otherwise it works great. Also tried setting c3p0's preferredQuery for ConnectionTester to "select 1;". Problem persists. Name: pool-15-thread-2 State: RUNNABLE Total blocked: 0 Total waited: 1 Stack trace: java.net.SocketInputStream.socketRead0(Native Method) java.net.SocketInputStream.read(SocketInputStream.java:150) java.net.SocketInputStream.read(SocketInputStream.java:121) java.io.BufferedInputStream.fill(BufferedInputStream.java:235) java.io.BufferedInputStream.read1(BufferedInputStream.java:275) java.io.BufferedInputStream.read(BufferedInputStream.java:334) - locked java.io.BufferedInputStream@5cec5796 nl.cwi.monetdb.mcl.net.MapiSocket$BlockInputStream._read(MapiSocket.java:846) nl.cwi.monetdb.mcl.net.MapiSocket$BlockInputStream.readBlock(MapiSocket.java:895) nl.cwi.monetdb.mcl.net.MapiSocket$BlockInputStream.read(MapiSocket.java:965) sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:283) sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:325) sun.nio.cs.StreamDecoder.read(StreamDecoder.java:177) - locked java.io.InputStreamReader@206a6883 java.io.InputStreamReader.read(InputStreamReader.java:184) java.io.BufferedReader.fill(BufferedReader.java:154) java.io.BufferedReader.readLine(BufferedReader.java:317) - locked java.io.InputStreamReader@206a6883 java.io.BufferedReader.readLine(BufferedReader.java:382) nl.cwi.monetdb.mcl.io.BufferedMCLReader.readLine(BufferedMCLReader.java:119) nl.cwi.monetdb.mcl.io.BufferedMCLReader.waitForPrompt(BufferedMCLReader.java:196) - locked nl.cwi.monetdb.mcl.io.BufferedMCLReader@29f38115 nl.cwi.monetdb.jdbc.MonetConnection$ResponseList.executeQuery(MonetConnection.java:2296) - locked nl.cwi.monetdb.mcl.net.MapiSocket@319f9f37 nl.cwi.monetdb.jdbc.MonetConnection$ResponseList.processQuery(MonetConnection.java:2274) nl.cwi.monetdb.jdbc.MonetStatement.internalExecute(MonetStatement.java:508) nl.cwi.monetdb.jdbc.MonetStatement.execute(MonetStatement.java:349) nl.cwi.monetdb.jdbc.MonetStatement.executeQuery(MonetStatement.java:525) com.mchange.v2.c3p0.impl.DefaultConnectionTester.activeCheckConnection(DefaultConnectionTester.java:73) com.mchange.v2.c3p0.impl.DefaultConnectionTester.statusOnException(DefaultConnectionTester.java:143) com.mchange.v2.c3p0.AbstractConnectionTester.statusOnException(AbstractConnectionTester.java:82) com.mchange.v2.c3p0.impl.NewPooledConnection.handleThrowable(NewPooledConnection.java:437) - locked com.mchange.v2.c3p0.impl.NewPooledConnection@3340302c com.mchange.v2.c3p0.impl.NewProxyStatement.executeUpdate(NewProxyStatement.java:78) com.sokrati.moneta.mclient.JdbcMonetdbClient.executeUpdate(JdbcMonetdbClient.java:154) com.sokrati.moneta.multiplexer.MonetdbClientCallable.call(MonetdbClientCallable.java:46) com.sokrati.moneta.multiplexer.MonetdbClientCallable.call(MonetdbClientCallable.java:1) java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) java.util.concurrent.FutureTask.run(FutureTask.java:166) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) java.lang.Thread.run(Thread.java:722) Regards, Tapomay. _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

2 6