thanks all!
On Wed, Aug 12, 2015 at 5:58 AM, Stefan Manegold
Hi,
Sjoerd's patch http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=92af99c05811 appears to work for me.
Thanks!
Stefan
----- On Aug 12, 2015, at 10:20 AM, Stefan Manegold Stefan.Manegold@cwi.nl wrote:
Hi,
while I so far failed to reproduce the behavior reliably, I just happen to have a Oct2014-generated database where upgrading to Jul2015 show the reported error, not only with the initial upgrade, but with every start of Jul2015 on that database.
Below you find a debug trace of the event, in case this helps some experts (Sjoerd?, Niels?) to understand / locate the problem ...
Thanks!
Stefan
======== (gdb) r Starting program: /export/scratch1/manegold/Monet/HG/Jul2015/prefix/_/bin/mserver5 --dbpath=/export/scratch1/manegold/dbfarm --dbextra=/export/scratch2/manegold/dbfarm [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". # MonetDB 5 server v11.21.0 # This is an unreleased version # Serving database 'dbfarm', using 8 threads # Compiled for x86_64-unknown-linux-gnu/64bit with 64bit OIDs and 128bit integers dynamically linked # Found 15.590 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2015 MonetDB B.V., all rights reserved # Visit http://www.monetdb.org/ for further information [New Thread 0x7ffff047c700 (LWP 15521)] [New Thread 0x7fffec658700 (LWP 15549)] # Listening for connection requests on mapi:monetdb://127.0.0.1:50000/ # MonetDB/GIS module loaded # Start processing logs sql/sql_logs version 52200
Breakpoint 1, GDKerror (format=0x7ffff778a8b8 "BATkey: void column cannot be unique.\n") at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/gdk/gdk_utils.c:1498 1498 size_t len = strlen(GDKERROR); Missing separate debuginfos, use: debuginfo-install R-core-3.2.0-2.fc20.x86_64 atlas-3.8.4-14.fc20.x86_64 blas-3.4.2-8.fc20.x86_64 bzip2-libs-1.0.6-9.fc20.x86_64 cfitsio-3.350-2.fc20.x86_64 cyrus-sasl-lib-2.1.26-14.fc20.x86_64 geos-3.4.2-1.fc20.x86_64 glibc-2.18-19.fc20.x86_64 gsl-1.15-9.fc20.x86_64 hdf5-1.8.11-6.fc20.x86_64 keyutils-libs-1.5.9-1.fc20.x86_64 krb5-libs-1.11.5-19.fc20.x86_64 libcom_err-1.42.12-4.fc20.x86_64 libcurl-7.32.0-20.fc20.x86_64 libgcc-4.8.3-7.fc20.x86_64 libgfortran-4.8.3-7.fc20.x86_64 libgomp-4.8.3-7.fc20.x86_64 libicu-50.1.2-12.fc20.x86_64 libidn-1.28-3.fc20.x86_64 libjpeg-turbo-1.3.1-3.fc20.x86_64 libquadmath-4.8.3-7.fc20.x86_64 libselinux-2.2.1-6.fc20.x86_64 libssh2-1.5.0-1.fc20.x86_64 libstdc++-4.8.3-7.fc20.x86_64 libuuid-2.24.2-2.fc20.x86_64 libxml2-2.9.1-4.fc20.x86_64 ncurses-libs-5.9-12.20130511.fc20.x86_64 netcdf-4.3.2-4.fc20.x86_64 nspr-4.10.8-1.fc20.x86_64 nss-3.18.0-1.fc20.x86_64 nss-softokn-freebl-3.18.0-1.fc20.x86_64 nss-util-3.18.0-1.fc! 20.x86_64 openldap-2.4.39-4.fc20.x86_64 openssl-libs-1.0.1e-42.fc20.x86_64 pcre-8.33-11.fc20.x86_64 readline-6.2-10.fc20.x86_64 xz-libs-5.1.2-12alpha.fc20.x86_64 zlib-1.2.8-3.fc20.x86_64 (gdb) bt #0 GDKerror (format=0x7ffff778a8b8 "BATkey: void column cannot be unique.\n") at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/gdk/gdk_utils.c:1498 #1 0x00007ffff7540db5 in BATkey (b=0x1ecda50, flag=1) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/gdk/gdk_bat.c:2162 #2 0x00007ffff73d97f4 in ALIGNsetH (b1=0x1ecda50, b2=0x60b810) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/gdk/gdk_align.c:117 #3 0x00007ffff752f841 in BATcopy (b=0x60b810, ht=0, tt=5, writable=1, role=1) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/gdk/gdk_bat.c:937 #4 0x00007ffff75590ae in logger_new (debug=0, fn=0x7fffef1faaaa "sql", logdir=0x7fffef1d55b0 "sql_logs", version=52200, prefuncp=0x7fffef1a6a82
, postfuncp=0x7fffef1a6b9f ) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/gdk/gdk_logger.c:1546 #5 0x00007ffff75598da in logger_create (debug=0, fn=0x7fffef1faaaa "sql", logdir=0x7fffef1d55b0 "sql_logs", version=52200, prefuncp=0x7fffef1a6a82 , postfuncp=0x7fffef1a6b9f ) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/gdk/gdk_logger.c:1720 #6 0x00007fffef1a8028 in bl_create (debug=0, logdir=0x7fffef1d55b0 "sql_logs", cat_version=52200) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/sql/storage/bat/bat_logger.c:156
#7 0x00007fffef187ddc in store_init (debug=0, store=store_bat, readonly=0, singleuser=0, logdir=0x7fffef1d55b0 "sql_logs", stk=0) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/sql/storage/store.c:1341 #8 0x00007fffef0f925d in mvc_init (debug=0, store=store_bat, ro=0, su=0, stk=0) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/sql/server/sql_mvc.c:36 #9 0x00007fffef041f91 in SQLinit () at
/ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/sql/backends/monet5/sql_scenario.c:251
#10 0x00007fffef041b39 in SQLprelude (ret=0x9bd8f0) at
/ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/sql/backends/monet5/sql_scenario.c:178
#11 0x00007ffff7b12834 in malCommandCall (stk=0x9bd690, pci=0x1dc39d0) at
/ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/monetdb5/mal/mal_interpreter.c:77
#12 0x00007ffff7b15871 in runMALsequence (cntxt=0x7ffff047d020, mb=0x68d110, startpc=1, stoppc=0, stk=0x9bd690, env=0x0, pcicaller=0x0) at
/ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/monetdb5/mal/mal_interpreter.c:655
#13 0x00007ffff7b142fb in runMAL (cntxt=0x7ffff047d020, mb=0x68d110, mbcaller=0x0, env=0x0) at
/ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/monetdb5/mal/mal_interpreter.c:365
#14 0x00007ffff7b44c1e in MALengine (c=0x7ffff047d020) at
/ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/monetdb5/mal/mal_session.c:637
#15 0x00007ffff7b4272e in malBootstrap () at
/ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/monetdb5/mal/mal_session.c:57
#16 0x00007ffff7aee041 in mal_init () at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/monetdb5/mal/mal.c:95 #17 0x0000000000403afc in main (argc=3, av=0x7fffffffda68) at
/ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/tools/mserver/mserver5.c:624
(gdb) li 1493 1494 void 1495 GDKerror(const char *format, ...) 1496 { 1497 char message[GDKERRLEN]; 1498 size_t len = strlen(GDKERROR); 1499 va_list ap; 1500 1501 if (!strncmp(format, GDKERROR, len)) { 1502 len = 0; (gdb) up #1 0x00007ffff7540db5 in BATkey (b=0x1ecda50, flag=1) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/gdk/gdk_bat.c:2162 2162 GDKerror("BATkey: void column cannot be unique.\n"); (gdb) li 2157 if (b->hseqbase != oid_nil && flag == FALSE) { 2158 GDKerror("BATkey: dense column must be unique.\n"); 2159 return GDK_FAIL; 2160 } 2161 if (b->hseqbase == oid_nil && flag == TRUE && b->batCount > 1) { 2162 GDKerror("BATkey: void column cannot be unique.\n"); 2163 return GDK_FAIL; 2164 } 2165 } 2166 if (flag) (gdb) p flag $1 = 1 (gdb) p b $2 = (BAT *) 0x1ecda50 (gdb) p *b $3 = {batCacheid = 909, H = 0x1ecda90, T = 0x1ecdb30, S = 0x1ecdbd0} (gdb) p b->H $4 = (COLrec *) 0x1ecda90 (gdb) p BATprint(b) #--------------------------# # h t # name # void int # type #--------------------------# [ nil, 1 ] [ nil, 0 ] $5 = GDK_SUCCEED (gdb) p *b->H $6 = {id = 0x7ffff77894e1 "h", width = 0, type = 0 '\000', shift = 0 '\000', varsized = 1, key = 0, dense = 0, nonil = 1, nil = 0, sorted = 1, revsorted = 1, align = 1028385, nokey = {0, 0}, nosorted = 0, norevsorted = 0, nodense = 0, seq = 9223372036854775808, heap = {free = 0, size = 0, base = 0x0, filename = 0x0, copied = 0, hashash = 0, forcemap = 0, storage = STORE_MEM, newstorage = STORE_MEM, dirty = 0 '\000', farmid = 1 '\001', parentid = 0}, vheap = 0x0, hash = 0x0, imprints = 0x0, props = 0x0} (gdb) p *b->T $7 = {id = 0x7ffff77894e3 "t", width = 4, type = 5 '\005', shift = 2 '\002', varsized = 0, key = 0, dense = 0, nonil = 1, nil = 0, sorted = 1, revsorted = 1, align = 1028386, nokey = {0, 0}, nosorted = 0, norevsorted = 0, nodense = 0, seq = 0, heap = {free = 8, size = 1024, base = 0x1ecdcf0 "\001", filename = 0x1ecdcc0 "16/1615.tail", copied = 0, hashash = 0, forcemap = 0, storage = STORE_MEM, newstorage = STORE_MEM, dirty = 0 '\000', farmid = 1 '\001', parentid = 0}, vheap = 0x0, hash = 0x0, imprints = 0x0, props = 0x0} (gdb) p *b->S $8 = {tid = 140737333930113, stamp = 69, copiedtodisk = 0, dirty = 1, dirtyflushed = 0, descdirty = 1, restricted = 0, persistence = 1, role = 1, unused = 0, sharecnt = 0, map_head = 0 '\000', map_tail = 0 '\000', map_hheap = 0 '\000', map_theap = 0 '\000', deleted = 0, first = 0, inserted = 0, count = 2, capacity = 256} (gdb) ========
----- On Aug 11, 2015, at 9:32 AM, Stefan Manegold Stefan.Manegold@cwi.nl wrote:
Well, in fact the error says that we internally try to mark a void
column >> unique, >> which is not allowed and hence not done. >> >> Thus, there seems to be a bug that we incorrectly try to mark a void column >> unique, >> but since that fails, this is likely more an "annoying" than a crucial bug ... >> >> Stefan >> >> ----- On Aug 11, 2015, at 9:29 AM, Stefan Manegold Stefan.Manegold@cwi.nl wrote: >> >>>> hi, >>>> does the notification itself indicate anything important? >>> >>> Internally incorrectly set (internal) BAT property. >>> That might be crucial as we (internally) use properties to "cut corners" (aka. >>> "optimize"). >>> >>>> if people do not flush the logs, will any data be lost? >>> >>> Should not --- the server flushes logs itself. >>> >>> I just suggested for the purpose of reproducing this problem, >>> to trigger the flush instead of waiting for the server to do it "eventually" >>> and transparently in the background ... >>> >>> Stefan >>> >>>> thanks! >>>> >>>> On Tue, Aug 11, 2015 at 2:42 AM, Stefan Manegold < Stefan.Manegold@cwi.nl > >>>> wrote: >>>> >>>> >>>> Hi, >>>> >>>> I've just seen similar behavior, >>>> but failed so far to reliably reproduce it with a minimal test. >>>> >>>> It seems requires at least >>>> creating a new DB with Oct2014, >>>> creating a table, inserting a least 2 tuples, >>>> stopping & restarting Oct2014 server to flush WAL logs, >>>> stopping Oct2014 server >>>> starting Jul2015 server to upgrade. >>>> >>>> Best, >>>> Stefan >>>> >>>> ----- On Aug 10, 2015, at 7:55 PM, Anthony Damico ajdamico@gmail.com wrote: >>>> >>>>> hi, i've just upgraded on my windows 8 laptop. every time i open the demo >>>>> server, i get this GDK error below. it doesn't appear to have any effect when i >>>>> run some simple queries. the error below does not appear when i create a fresh >>>>> dbfarm elsewhere, it only happened on a database that's been upgraded. >>>>> >>>>> any idea what's going on? thanks! >>>>> >>>>> >>>>> # MonetDB 5 server v11.21.1 "Jul2015" >>>>> # Serving database 'demo', using 8 threads >>>>> # Compiled for x86_64-pc-winnt/64bit with 64bit OIDs dynamically linked >>>>> # Found 15.914 GiB available main-memory. >>>>> # Copyright (c) 1993-July 2008 CWI. >>>>> # Copyright (c) August 2008-2015 MonetDB B.V., all rights reserved >>>>> # Visit http://www.monetdb.org/ for further information >>>>> # Listening for connection requests on mapi:monetdb:// 127.0.0.1:50000/ >>>>> # Start processing logs sql/sql_logs version 52200 >>>>> # Start reading the write-ahead log 'sql_logs\sql\log.14' >>>>> # Finished reading the write-ahead log 'sql_logs\sql\log.14' >>>>> # Finished processing logs sql/sql_logs >>>>> # MonetDB/SQL module loaded >>>>> !GDKerror:!ERROR: BATkey: void column cannot be unique. >>>>> !ERROR: BATkey: void column cannot be unique. >>>>>> >>>>> >>>>> >>>>> >>>>> On Mon, Aug 10, 2015 at 9:25 AM, Sjoerd Mullender < sjoerd@acm.org > wrote: >>>>> >>>>> >>>>> -----BEGIN PGP SIGNED MESSAGE----- >>>>> Hash: SHA256 >>>>> >>>>> I have created a release candidate for the Jul2015 release. The >>>>> candidate is available at http://dev.monetdb.org/downloads/testing/ . >>>>> If all goes well, this candidate will morph into a release sometime >>>>> soon(ish). In the mean time, you can use this release candidate and >>>>> test to see whether it works. And if it doesn't work, I would really >>>>> like to hear about it. >>>>> >>>>> - -- >>>>> Sjoerd Mullender >>>>> -----BEGIN PGP SIGNATURE----- >>>>> Version: GnuPG v2 >>>>> >>>>> iQEcBAEBCAAGBQJVyKZgAAoJEISMxT6LrWYgE28H/2rudFXhcIfNgnX4yYJlpRT5 >>>>> eI2Dzp9kShEmRibvNkWvCEF1j/qyWgQhx/jlHvucuEF/0k+U+7NYuK0vhzA0llyJ >>>>> hspMkQ73UDXeVO0/SivtNYb40UpMyUQoXC7nSaSneglVDe4JcX4S3Y5GvHPByiqz >>>>> 3UNMGDARfWlqPcbTtW57XfLejK/sd8nNZ3mGQuerVqEvMcKZaZIBSXjg/16WQVsN >>>>> DMbPacPpu3yFeWQQtP2er6SL3UdfwgOP5+VNifKyjW9i6ZtB2lHWLEBWQY8G/5Tb >>>>> 5lwgHvD99SJeLRMzcW5TFWhwv0ROx8chKP3btxMofZxjHhqseswGRLNDQa4XCU0= >>>>> =ktpg >>>>> -----END PGP SIGNATURE----- >>>>> _______________________________________________ >>>>> users-list mailing list >>>>> users-list@monetdb.org >>>>> https://www.monetdb.org/mailman/listinfo/users-list >>>>> >>>>> >>>>> _______________________________________________ >>>>> users-list mailing list >>>>> users-list@monetdb.org >>>>> https://www.monetdb.org/mailman/listinfo/users-list >>>> >>>> -- >>>>| Stefan.Manegold@CWI.nl | DB Architectures (DA) | >>>>| www.CWI.nl/~manegold/ | Science Park 123 (L321) | >>>>| +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) | >>>> _______________________________________________ >>>> users-list mailing list >>>> users-list@monetdb.org >>>> https://www.monetdb.org/mailman/listinfo/users-list >>>> >>>> >>>> _______________________________________________ >>>> users-list mailing list >>>> users-list@monetdb.org >>>> https://www.monetdb.org/mailman/listinfo/users-list >>> >>> -- >>>| Stefan.Manegold@CWI.nl | DB Architectures (DA) | >>>| www.CWI.nl/~manegold/ | Science Park 123 (L321) | >>> | +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) | >> >> -- >>| Stefan.Manegold@CWI.nl | DB Architectures (DA) | >>| www.CWI.nl/~manegold/ | Science Park 123 (L321) | >>| +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) | >> _______________________________________________ >> users-list mailing list >> users-list@monetdb.org >> https://www.monetdb.org/mailman/listinfo/users-list > > -- >| Stefan.Manegold@CWI.nl | DB Architectures (DA) | >| www.CWI.nl/~manegold/ | Science Park 123 (L321) | >| +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) | > _______________________________________________ > users-list mailing list > users-list@monetdb.org > https://www.monetdb.org/mailman/listinfo/users-list
-- | Stefan.Manegold@CWI.nl | DB Architectures (DA) | | www.CWI.nl/~manegold/ | Science Park 123 (L321) | | +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) | _______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list