[MonetDB-users] Windows JDBC Issues
Hello (again!), I have MonetDB running on a 32-bit Windows Vista box. When I start the server it says it's "MonetDB server v5.10.2, based on kernel v1.28.2". I'm using JDBC batch updates to populate about 100 tables with approximately 4 million records per table. I've a couple of issues: 1) My batch updates fail from time-to-time. On average this might be every 10,000 batch updates with each batch containing 100 insertions. I get an exception with the following message: "Error(s) occurred while executing the batch, see next SQLExceptions for details" If I close the connection then open another one I can continue quite happily. If I try to reuse the existing connection with a cleared batch I get the same error every time until I do close the connection. I think I may not be the only person with this issue: http://wiki.pentaho.com/display/EAI/MonetDB http://wiki.pentaho.com/display/EAI/MonetDB 2) After a very large amount of insertions (more like 100+ million records, across numerous tables) the insertion simply hangs (a commit usually takes about a second and I've left it 30+ minutes). After killing the client the server becomes impossible to connect to. Even after a reboot I get the following messages: SERVER:
!ERROR: Incompatible database version 000000, this server supports version 050000 !ERROR: Please move away c:\data\monetdbdata\MonetDB5\sql_logs\demo\sql\ and its corresponding dbfarm. !mvc_init: unable to create system tables
CLIENT: !SQLException:SQLinit:Catalogue initialization failed JDBC CLIENT: Connection reset (Mserver still alive?) If, however, I delete/move the sql_logs directory then restart things then I can connect and my data is still there. The last time I did this one of the log files (log.97) was ~250mb in size. Any help, advice or comments would be much appreciated. I'm going to spend this afternoon trying out the bulk COPY INTO functionality. Thanks, Scott -- View this message in context: http://www.nabble.com/Windows-JDBC-Issues-tp23196456p23196456.html Sent from the monetdb-users mailing list archive at Nabble.com.
Hi! On 23-04-2009 06:03:19 -0700, gsd wrote:
I have MonetDB running on a 32-bit Windows Vista box. When I start the server it says it's "MonetDB server v5.10.2, based on kernel v1.28.2". I'm using JDBC batch updates to populate about 100 tables with approximately 4 million records per table. I've a couple of issues:
1) My batch updates fail from time-to-time. On average this might be every 10,000 batch updates with each batch containing 100 insertions. I get an exception with the following message: "Error(s) occurred while executing the batch, see next SQLExceptions for details"
Does your tool allow to extract the real problem by following the SQLException chain? That would help a lot to identify the batch fails.
If I close the connection then open another one I can continue quite happily. If I try to reuse the existing connection with a cleared batch I get the same error every time until I do close the connection.
I think I may not be the only person with this issue: http://wiki.pentaho.com/display/EAI/MonetDB http://wiki.pentaho.com/display/EAI/MonetDB
2) After a very large amount of insertions (more like 100+ million records, across numerous tables) the insertion simply hangs (a commit usually takes about a second and I've left it 30+ minutes). After killing the client the server becomes impossible to connect to. Even after a reboot I get the following messages:
SERVER:
!ERROR: Incompatible database version 000000, this server supports version 050000 !ERROR: Please move away c:\data\monetdbdata\MonetDB5\sql_logs\demo\sql\ and its corresponding dbfarm. !mvc_init: unable to create system tables
CLIENT: !SQLException:SQLinit:Catalogue initialization failed
JDBC CLIENT: Connection reset (Mserver still alive?)
If, however, I delete/move the sql_logs directory then restart things then I can connect and my data is still there. The last time I did this one of the log files (log.97) was ~250mb in size.
Any help, advice or comments would be much appreciated. I'm going to spend this afternoon trying out the bulk COPY INTO functionality.
Hmmm, this sounds unrelated to JDBC.
Thanks for the reply Fabian. Fabian Groffen wrote:
Does your tool allow to extract the real problem by following the SQLException chain? That would help a lot to identify the batch fails.
It does now. The SQLException chain has three levels: Message: "Error(s) occurred while executing the batch, see next SQLExceptions for details" State: null Code: 0 Message: "current transaction is aborted (please ROLLBACK)" (repeated 91 times) State: null Code: 0 Message: "current transaction is aborted (please ROLLBACK)" (repeated 8 times) State: null Code: 0 This doesn't seem very informative to me... I'm using version 6 of Java - is there a suggested version to use or versions not to use? Finally, on a different note, is it possible to pipe STDIN to COPY INTO calls via JDBC => MonetDb? Thanks again, Scott -- View this message in context: http://www.nabble.com/Windows-JDBC-Issues-tp23196456p23212225.html Sent from the monetdb-users mailing list archive at Nabble.com.
On 24-04-2009 01:37:07 -0700, gsd wrote:
Thanks for the reply Fabian.
Fabian Groffen wrote:
Does your tool allow to extract the real problem by following the SQLException chain? That would help a lot to identify the batch fails.
It does now. The SQLException chain has three levels:
Message: "Error(s) occurred while executing the batch, see next SQLExceptions for details" State: null Code: 0
Message: "current transaction is aborted (please ROLLBACK)" (repeated 91 times) State: null Code: 0
Message: "current transaction is aborted (please ROLLBACK)" (repeated 8 times) State: null Code: 0
This is our error message for a transaction violation (concurrency conflict). This suggests that you are doing parallel work here, or at least have more than one client connected and doing something. Note that MonetDB/SQL only implements fully serialisable transactions, the most strict transaction isolation level.
This doesn't seem very informative to me... I'm using version 6 of Java - is there a suggested version to use or versions not to use?
There is/was a bug in Java 6 that causes an IndexOutOfBoundsException somewhere deep down at the driver's kernel, but since you don't hit that error, I think it's fine. Java 5 (1.5) is still the J2EE default, and our driver is also at that level.
Finally, on a different note, is it possible to pipe STDIN to COPY INTO calls via JDBC => MonetDb?
I think it's technically possible to send COPY INTO + the data following over JDBC, but as JDBC is statement based, this does not align very much with the interface.
participants (2)
-
Fabian Groffen -
gsd