Dear all,
I am currently testing an application running on MonetDB that experiences intermittent errors when under high concurrency.
I believe these are aborted transactions (the mapi error code is -4).
The applications loads a small input table using the mapi interface, runs two queries internally (including writing to an intermediate table) and then reads the result set.
I have tried to produce a similar minimal example using the Python API which does produce concurrency errors and I am wondering what the correct thing to do on the client side is. Should I simply resubmit the failed query?
The Python script used is shown below:
import monetdb.sql
import sys
from multiprocessing import Process
import uuid
def f():
dbh = monetdb.sql.Connection(port=int(sys.argv[1]),database=sys.argv[2],hostname=sys.argv[3],autocommit=True)
for i in range (0,20):
tablename = "input_"+str(uuid.uuid1()).replace("-","_")
cursor = dbh.cursor()
#cursor.execute('create local temporary table %s (term int, p float) on commit preserve rows;' % tablename)
cursor.execute('declare table %s (term int, p float);' % tablename)
cursor.execute('insert into %s values (1,0.1);' % tablename)
cursor.execute('select * from %s;' % tablename)
print(cursor.fetchall())
cursor.execute('drop table %s;' % tablename)
dbh.close()
procs=[]
for t in range(0,int(sys.argv[4])):
p = Process(target=f)
procs.append(p)
p.start()
for proc in procs:
proc.join()
Running this locally with "python2.7 concurrency.py 50000 test_db 127.0.0.1 10" results in a number of error messages like:
ProgrammingError: 40000!COMMIT: transaction is aborted because of concurency conflicts, will ROLLBACK instead
What is the right thing to do? It seems that anything that writes to the db even when they are separate tables causes these messages.
Best regards and thanks,