Re: [MonetDB-users] Serious concurrency issues via python api

3 Sep 2009

      On Wed, Sep 02, 2009 at 11:06:12PM +0200, Peter Boncz wrote:
...
Hi,
Snapshot isolation in principle only needs to abort write-write conflicts.
However, MonetDB aims at serializabilty. Maybe we should consider offering SI
Yes MonetDB has serializability. Still a single only read transaction 
shouldn't affect a writing transaction.
Niels
...
as well?
In addition to that, my advise has been been for some time to consider
row-wise write-write conflict checking rather than full-table.
Peter
...
If the write is Isolated from a read then should a read really be allowed to
cause a concurrency failure on the write?  The read only needs to be
guaranteed to not see the table in a partial/incomplete state and the
write's changes are isolated until it commits.   A read conflicting with a
write seems like something altogether different than a write conflicting
with another write.
Am I missing something here?
73,
Matthew W. Jones (KI4ZIB)
http://matburt.net
On Wed, Sep 2, 2009 at 4:16 PM, Stefan Manegold wrote:
...
...
Are you saying that if someone reads data from a table while another
transaction is bulk-loading data that that will cause a concurrency
conflict
and a forced rollback?
I'm looking at the behavior of one of my long running batch loads and
On Wed, Sep 02, 2009 at 04:03:17PM -0400, Matthew Jones wrote:
that
...
looks to be the case... if so, that seems pretty extreme.   I could
understand protecting from simultaneous modification but it seems wrong
to
cause a write to fail for any read on a table.
writes conflict with reads, and reads with writes, just as writes conflict
with writes; only multiple concurrent reads do not conlict with each other.
Stefan
...
73,
Matthew W. Jones (KI4ZIB)
http://matburt.net
On Wed, Sep 2, 2009 at 3:41 PM, Martin Kersten 
...
Matthew Jones wrote:
...
There's no relational data in these tables, I guess I just need
something internal to manage concurrency... like blocking table locks
so
I don't have to wait for a failure in order to manage concurrency on
my
own.
you either have to serialize the updates/access at your client
interface,
or hook into the MAL kernel to get access to the general lock/semaphore
module.
...
I noticed in one of my operations  which was a single update to a
...
that no other processes were using that it gave me that error,
informed
Optimistic concurrency control should also ensure that non of the
concurrent
table
transactions reads stale data.
This becomes tricky if you have multiple user sessions with both
autocommit
and compound transactions, because then the timespans to trigger
a transaction failure are larger.
...
me of the rollback, but the changes still took effect.... which is
alarming since I will now need to:
1) Try the operation
2) If it fails, check to see if the change actually took effect
3) if the changes didn't take effect re-try the operation
This is disconcerting.
73,
Matthew W. Jones (KI4ZIB)
http://matburt.net
On Wed, Sep 2, 2009 at 3:02 PM, Martin Kersten <
Martin.Kersten@cwi.nl
mailto:Martin.Kersten@cwi.nl> wrote:
Dear Matthew,
A quick initial reaction. MonetDB uses an optimistic concurrency
control
    scheme, which is geared at high-read, low-write applications. If
you
hit
    the system with multiple concurrent writes on the same table,
then
    indeed
    to secure acid properties, transactions are bound to fail.
MonetDB
    indeed
    conforms to the semantics of a transaction safe system.
Blocking queries is equivalent to relation level locking.
    This is not the preferred way to go for the kernel, because
someone
    will immediately ask for more granularities and als relaxation of
    the ACID principles.
There are several application scenarios to deal with this issue,
    but all depend on the semantics required.
regards, Martin
Matthew Jones wrote:
     > I have multiple processes reading and writing to the same
table
    and I've
     > noticed really persistent concurrency problems:
     >
     >   File
"/usr/lib/python2.5/site-packages/monetdb/sql/cursors.py",
    line
     > 208, in execute
     >     self.__store_result(block)
     >   File
"/usr/lib/python2.5/site-packages/monetdb/sql/cursors.py",
    line
     > 520, in __store_result
     >     self.__exception_handler(InterfaceError, "Unknown state,
%s"
    % block)
     >   File
"/usr/lib/python2.5/site-packages/monetdb/sql/cursors.py",
    line
     > 574, in __exception_handler
     >     raise exception_class(message)
     > InterfaceError: Unknown state, &2 65541 -1
     > !ERROR: COMMIT: transaction is aborted because of concurency
    conflicts,
     > will ROLLBACK instead
     >
     > I'm not sure what the problem is, but it exists in such a way
    that all
     > operations that load or update information in MonetDB have to
be
put
     > into a loop to continuously retry.
     >
     > I'm using autocommit, but I would expect MonetDB to handle
    concurrency
     > issues in a decent way, like maybe blocking a query until the
other
     > query finished.  I'm not clobbering data, most of these are
just
bulk
     > load operations that insert new rows.
     >
     > Is there any way to address this issue without having to
putting
    retry
     > loops around all of the code I'm writing to write to the
database?
     >
     > 73,
     > Matthew W. Jones (KI4ZIB)
     > http://matburt.net
     >
     >
     >

...
...
...
>
     >

...
...
...
and focus on
     > what you do best, core application coding. Discover what's new
with
...
> Crystal Reports now.  http://p.sf.net/sfu/bobj-july
     >
     >
     >
...
...
...
> Let Crystal Reports handle the reporting - Free Crystal
Reports
...
2008 30-Day
     > trial. Simplify your report design, integration and deployment

...
...
...
>
     > _______________________________________________
     > MonetDB-users mailing list
     > MonetDB-users@lists.sourceforge.net
    mailto:MonetDB-users@lists.sourceforge.net
     > https://lists.sourceforge.net/lists/listinfo/monetdb-users

...
...
...
Let Crystal Reports handle the reporting - Free Crystal Reports
2008
...
30-Day
    trial. Simplify your report design, integration and deployment -
and
...
focus on
    what you do best, core application coding. Discover what's new
with
...
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
    _______________________________________________
    MonetDB-users mailing list
    MonetDB-users@lists.sourceforge.net
    mailto:MonetDB-users@lists.sourceforge.net
    https://lists.sourceforge.net/lists/listinfo/monetdb-users

...
...
...

...
...
...
Let Crystal Reports handle the reporting - Free Crystal Reports 2008
30-Day
trial. Simplify your report design, integration and deployment - and
focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july

...
...
...
_______________________________________________
MonetDB-users mailing list
MonetDB-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/monetdb-users

...
...
Let Crystal Reports handle the reporting - Free Crystal Reports 2008
30-Day
trial. Simplify your report design, integration and deployment - and
focus
on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
MonetDB-users mailing list
MonetDB-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/monetdb-users
...
------------------------------------------------------------------------------
...
Let Crystal Reports handle the reporting - Free Crystal Reports 2008
30-Day
trial. Simplify your report design, integration and deployment - and
focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
MonetDB-users mailing list
MonetDB-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/monetdb-users
--
| Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl |
| CWI,  P.O.Box 94079 |
http://www.cwi.nl/~manegold/http://www.cwi.nl/%7Emanegold/ |
| 1090 GB Amsterdam   | Tel.: +31 (20) 592-4212       |
| The Netherlands     | Fax : +31 (20) 592-4312       |
------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus
on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
MonetDB-users mailing list
MonetDB-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/monetdb-users
------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.
http://p.sf.net/sfu/bobj-july_______________________________________________
MonetDB-users mailing list
MonetDB-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/monetdb-users
------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day 
trial. Simplify your report design, integration and deployment - and focus on 
what you do best, core application coding. Discover what's new with 
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
MonetDB-users mailing list
MonetDB-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/monetdb-users
-- 
Niels Nes, Centre for Mathematics and Computer Science (CWI)
Science Park 123, 1098 XG Amsterdam, The Netherlands
room C0.02/M3.46,  phone ++31 20 592-4098, fax ++31 20 592-4312
url: http://www.cwi.nl/~niels   e-mail: Niels.Nes@cwi.nl