Re: Multi threaded (2 threads) Java program for copy command
Hi Martin,
Thanks for your prompt reply , I am using two different tables here but still I am getting "Exception in thread "main" java.sql.SQLException: COMMIT: failed" exception.
Message: 2 Date: Thu, 12 Mar 2015 08:32:31 +0100 From: Martin Kersten
To: users-list@monetdb.org Subject: Re: Multi threaded (2 threads) Java program for copy command commit fails for one thread Message-ID: <5501410F.1070508@monetdb.org> Content-Type: text/plain; charset=windows-1252; format=flowed Hi,
Yes. Insertion into the same table would be invalidated due to the transaction semantics. See https://www.monetdb.org/Documentation/Manuals/SQLreference/Transactions
Hi All,
I am evaluating monetdb for our dwh and I am trying to load data from two different csv file to 2 different tables using multi-threaded java
On 12/03/15 08:18, Shivanandan Gupta wrote: program in parallel, for one thread the load is successful and other one is failing with exception given below.
Exception in thread "main" java.sql.SQLException: COMMIT: failed at
nl.cwi.monetdb.jdbc.MonetConnection$ResponseList.executeQuery(MonetConnection.java:2535)
at nl.cwi.monetdb.jdbc.MonetConnection$ResponseList.processQuery(MonetConnection.java:2284) at nl.cwi.monetdb.jdbc.MonetConnection.commit(MonetConnection.java:374) at connect_monetdb.runETL(connect_monetdb.java:78) at connect_monetdb.main(connect_monetdb.java:14)
command used is :
String csql = "copy into s_asset_"+filename+" from '"+filepath+""+ filename +".csv' using delimiters',';"; st.execute(csql); con.commit();
I am using monetdb jdbc driver and monetdb server is also on the same machine.
Any help will be really great.
Thank you.
Cheers, Shivanandan Gupta
_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
On Mar 12, 2015, at 19:27, Shivanandan Gupta
wrote: Hi Martin,
Thanks for your prompt reply , I am using two different tables here but still I am getting "Exception in thread "main" java.sql.SQLException: COMMIT: failed" exception.
Hello Shivanandan, Concurrent insertion into two different tables should work. Do you have anymore information about why the COMMIT has failed? Maybe the problem lies somewhere else than concurrent insertions? - Have you tried the two COPY INTO separately? To know if the problem is indeed caused by concurrent insertions. - Does the problem occur one-time, or every time you repeatedly execute your code? Regards, Jennie
Message: 2 Date: Thu, 12 Mar 2015 08:32:31 +0100 From: Martin Kersten
To: users-list@monetdb.org Subject: Re: Multi threaded (2 threads) Java program for copy command commit fails for one thread Message-ID: <5501410F.1070508@monetdb.org> Content-Type: text/plain; charset=windows-1252; format=flowed Hi,
Yes. Insertion into the same table would be invalidated due to the transaction semantics. See https://www.monetdb.org/Documentation/Manuals/SQLreference/Transactions
On 12/03/15 08:18, Shivanandan Gupta wrote:
Hi All,
I am evaluating monetdb for our dwh and I am trying to load data from two different csv file to 2 different tables using multi-threaded java program in parallel, for one thread the load is successful and other one is failing with exception given below.
Exception in thread "main" java.sql.SQLException: COMMIT: failed at nl.cwi.monetdb.jdbc.MonetConnection$ResponseList.executeQuery(MonetConnection.java:2535) at nl.cwi.monetdb.jdbc.MonetConnection$ResponseList.processQuery(MonetConnection.java:2284) at nl.cwi.monetdb.jdbc.MonetConnection.commit(MonetConnection.java:374) at connect_monetdb.runETL(connect_monetdb.java:78) at connect_monetdb.main(connect_monetdb.java:14)
command used is :
String csql = "copy into s_asset_"+filename+" from '"+filepath+""+ filename +".csv' using delimiters',';"; st.execute(csql); con.commit();
I am using monetdb jdbc driver and monetdb server is also on the same machine.
Any help will be really great.
Thank you.
Cheers, Shivanandan Gupta
_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
Hi Jennie,
Thanks for your reply. please find my replies below:
Jennie Commented: Concurrent insertion into two different tables should
work. Do you have anymore information about why the COMMIT has failed?
Maybe the problem lies somewhere else than concurrent insertions?
Shiva's Comment: yes when I run two batch file by clicking it with
difference of second ( as it takes few seconds to click the second batch
file) and it runs perfectly, even I have tried running copy command
directly on the monetsql client and on two clients in parallel it works
perfectly. Is there some configuration change required on monetdb server?
Jennie Commented: - Have you tried the two COPY INTO separately? To know
if the problem is indeed caused by concurrent insertions.
Shiva's Comment: When I am trying with two separate java program running
concurrently its throwing same exception.
Jennie Comment: - Does the problem occur one-time, or every time you
repeatedly execute your code?
Shiva's Comment: Yes the same error message is coming every time I run
concurrent java programs with copy command in it ( here copying data to
different tables ).
Any help will be really appreciated.
Thank you.
Shivanandan Gupta.
Date: Fri, 13 Mar 2015 15:36:23 +0800
From: Ying Zhang
On Mar 12, 2015, at 19:27, Shivanandan Gupta
wrote: Hi Martin,
Thanks for your prompt reply , I am using two different tables here but still I am getting "Exception in thread "main" java.sql.SQLException: COMMIT: failed" exception.
Hello Shivanandan, Concurrent insertion into two different tables should work. Do you have anymore information about why the COMMIT has failed? Maybe the problem lies somewhere else than concurrent insertions? - Have you tried the two COPY INTO separately? To know if the problem is indeed caused by concurrent insertions. - Does the problem occur one-time, or every time you repeatedly execute your code? Regards, Jennie On Thu, Mar 12, 2015 at 4:57 PM, Shivanandan Gupta < shivnandangupta@gmail.com> wrote:
Hi Martin,
Thanks for your prompt reply , I am using two different tables here but still I am getting "Exception in thread "main" java.sql.SQLException: COMMIT: failed" exception.
Message: 2 Date: Thu, 12 Mar 2015 08:32:31 +0100 From: Martin Kersten
To: users-list@monetdb.org Subject: Re: Multi threaded (2 threads) Java program for copy command commit fails for one thread Message-ID: <5501410F.1070508@monetdb.org> Content-Type: text/plain; charset=windows-1252; format=flowed Hi,
Yes. Insertion into the same table would be invalidated due to the transaction semantics. See https://www.monetdb.org/Documentation/Manuals/SQLreference/Transactions
participants (2)
-
Shivanandan Gupta
-
Ying Zhang